Building Test Scenarios Archives | Page 2 of 2 | LoadStorm

Our verification process works exactly like Google Analytics. The idea is to prove it is your server so LoadStorm isn’t used for a denial of service attack. You can put a file with the name generated by LS into your root directory (doesn’t matter what is in it). So let’s say your target server is http://www.xyzcorp.com, then there should be a valid response if a request comes to your server for http://www.xyzcorp.com/loadstorm-90630fc588.html.

Alternatively, if you don’t want to use the file you can embed the code in your default page. To continue the example, the HTML page received from requesting http://www.xyzcorp.com should contain a string somewhere within it (inside a comment tag is good) that matches “loadstorm-90630fc588″. LoadStorm will parse the page returned by your server exactly like a browser will. In that parsing, the verification string must appear somewhere for us to consider it verified.

Once you have the file or text in place, click on the Build Tab, then select the plan name that is targeting your server. You will see a Verify Now link in the yellow shaded section that shows your verification code. Click on that link and you should now see “Verified: Yes” on the plan page. If you have difficulty or need assistance, please contact [email protected].

Please only use the domain name when adding a server, and only use the domain name once. Adding a server for http://www.xyz.com works great and verification is successful. Then creating a server for http://www.xyz.com/myfolder will not work and trying to verify it will produce an error.

Most people create several scenarios to accurately reflect different types of users. Scenarios execute simultaneously in order to hit your site with the correct mix of traffic. You control the weighting of each scenario by giving it a number relative to the others. For example, if you have 2 scenarios and you give one a weighting of 4 and the other a weighting of 1, then the first will generate 80% of the traffic and the second 20%. This 80/20 could be realistic for an retail e-commerce site where most of the people (80%) are browsing the product catalog, while a minority of people (20%) are going through a buying experience. You can of course throw in a low-weight scenario to represent some internal employees adding products or issuing refunds.

There is no limit to the number of scenarios you can create in a test plan. We encourage you to build as many as necessary to accurately reflect the real traffic patterns on your site. That will give you the most meaningful metrics for tuning your applications.

We also simulate the user “think time” with random pauses between steps. Real users click on something, then read the page or fill in a form, and then click on something else. Thus, you don’t want to have a simulated virtual user clicking on something every .5 seconds – it would not give you meaningful results if you want to see how your site holds up in the real world.

Also see: Virtual User Calculations about calculating the amount of virtual users you need for a realistic test.

It helps you with performance engineering. We recommend that you do not begin a large test at the maximum number of virtual users. Rather than run a test for 10 minutes beginning with 5,000 and ending with 5,000, we suggest that you perhaps start with 500 users and increase to 5,000 over an hour. This approach has the advantage of allowing your system to get all the parts working properly at smaller load (e.g. caches, threads, database connections) before the heavier volume starts exposing potential bottlenecks in your application.

From what we’ve seen in thousands of load tests with LoadStorm, it is common to see useful patterns in the metrics before the peak traffic is reached, and these metrics point to areas of performance limitations that can be tuned.

One of the primary reasons we price LoadStorm subscriptions to allow unlimited test runs in the month is because we know that load testing is invariably coupled with performance tuning – an iterative process of test/tune/test/tune. Therefore, we recommend you ramp the volume in order to get more value from the test results.

Yes, there are two additional ways for you to white list our traffic:

1. The user agent in the http header includes LoadStorm. However, if your test scenarios contain steps with https requests, then those headers will be encrypted.
2. If you are not testing against your production server, we recommend trying a non-standard port number to identify the traffic as load testing. LoadStorm allows you to use any port number and can be designated for each target server.

LoadStorm follows links when the session ID is appended to the URL query string because it searches for the text between the open and close anchor tags. If it doesn’t find that text, then the fall back is to match the XPATH. We stored the XPATH when the scenario was created (essentially an address inside the page’s DOM). The XPATH is a unique identifier that allows us to click on the anchor such as an image or the href attribute or execute the Javascript.

Yes, you may submit forms such as a login form and fill-in the fields (e.g. username, password) with data from your system.

First, you export a CSV file of user information from your user table(s). The first row of the CSV must contain the names of the columns, and these column names will be mapped to the field name in the step that submits the form.

The CSV file is uploaded by clicking on the UPLOAD DATA button on the Build tab. This button will be near the bottom, under the Form Data Sets section of the page. You must give the file a label so that you can choose that file to be used at the scenario level.

If you need to embed a comma inside a particular data field, simply place quotation marks around all field data in the file.

The LoadStorm Tour has a video tutorial showing how to create a login form submission.

The Form Data Set field that you see on the Scenario settings page is a mechanism to select an input source for form submission. Using the Form Data Sets functionality, it is possible to submit forms such as a login form and fill-in the fields (e.g. username, password) with data from your system. Other common types of form submission are search terms, credit card numbers, and profile preferences. Any type of form field that can be filled in with text can utilize the Form Data Sets.

You may upload your own data to be used for form submission, or you may use one of our built-in form data sets.

The Tiny, Small, and Huge form data sets are built-into LoadStorm. The only real difference between the three files are the number of rows. Tiny only has 10 records. Small has 100, and Huge has 10,000 records of user information. They contain user information:

• First Name
• Last Name
• Username
• Password
• Email address

Many of our customers use one of our built-in form data sets for registration on their web application or for logging in virtual users.

If you are going to upload your own data into LoadStorm, here is an example of the process for getting user credentials from your system to LoadStorm:

1. Export a CSV file of user information from your user table(s).
2. The first row of the CSV must contain the names of the columns, and these column names will be mapped to the field name in the step that submits the form.
3. The CSV file is uploaded by clicking on the UPLOAD DATA button on the Build tab. This button will be near the bottom, under the Form Data Sets section of the page.
4. You must give the file a label so that you can choose that file to be used at the scenario level.

Once you have the CSV data in LoadStorm (you should see it on the Build tab under the heading Form Data Sets), then you need to edit the scenario and choose the appropriate data set based on the label you assigned when uploading it. This essentially connects the CSV file to your scenario. Then when you add a step for form submission, you will be able to map the fields in the form to the columns of data in the Form Data Set.

The LoadStorm Tour has a video tutorial showing how to create a login form submission. Watch the video on YouTube

Here are some thoughts on increasing RPS in LoadStorm tests:

1. Reducing pause time between steps (at scenario level) – minimum is 10 second pause, so set both min & max to 10.
2. Increase the number of concurrent users.
3. Reduce the response time from your server which decreases LoadStorm’s waiting for a response.

RPS will always be impacted by the types of pages/resources your scenarios are requesting. The way it works is that LoadStorm cranks up the starting concurrent users shortly after test starts. It then walks each user through the scenario assigned to it. One step is executed (such as home page), and all the images, JS, CSS, XML are requested separately. When those responses are all returned, then the system pauses a random time between min & max pause, then it requests the next step. That continues until the end of the scenario, whereupon that virtual user is retired. After a few seconds, another vuser is created and the process starts again. So, having more users going at the same time, without waiting more than a few hundred ms to get a response, will increase RPS.

We have a mechanism to “import” URLs, and we have a video tutorial that may be helpful – http://youtu.be/5APaPn2zo6M

The concept is to upload a file of strings that can be substituted in a single step of a scenario. Most customers use this functionality to append a query string on the end of a domain. You can produce a CSV with one or more columns containing the strings. Then you add a step that opens a page, and there is specific syntax for mapping the string from a column of the CSV into the URL requested by the step.

It may be helpful to review the Here’s an example of what would be used for the step:

http://cia-factbook.s3.amazonaws.com#{FileOfPaths.pathFull1}

The hash and open brace are significant because they tell LoadStorm to substitute the text. “FileOfPaths” is the label for the CSV I uploaded into my LoadStorm account. “pathFull1″ is the column name inside FileOfPaths that contains the string I want to append on the domain.

To upload the CSV (form data set) of your URLs/query strings into LoadStorm:

1. Export a CSV file with one or more columns containing the URLs.
2. The first row of the CSV must contain the names of the columns, and these column names will be mapped to the substitution point in the step indicated by the #{ } delimiters.
3. The CSV file is uploaded by clicking on the UPLOAD DATA button on the Build tab. This button will be near the bottom, under the Form Data Sets section of the page.
4. You must give the file a label so that you can choose that file to be used at the scenario level.

Once you have the CSV data in LoadStorm (you should see it on the Build tab under the heading Form Data Sets), then you need to edit the scenario and choose the appropriate data set based on the label you assigned when uploading it. This essentially connects the CSV file to your scenario. Then when you add a step for using the URL/query string, you will be able to place #{formDataSet.columnName} from the Form Data Set.

When adding a step in LoadStorm, our emulator makes the request to your web application and receives the response containing the HTML code for the page. Our tool then parses the HTML to determine what additional resources (e.g. images, Javascript libraries) need to be requested for that page.

Each of those resources are requested and received. LoadStorm saves all of the HTML and other resources in our database along with other information it needs to reproduce the sequence of events for this step during a load test. The visual rendering you see at the bottom of the step is an iFrame where our tool places a copy of what was received during the step creation process. It is NOT an accurate rendering as you would see it in a full browser without all of the other LoadStorm code and process
“wrapped around” it. Essentially, what you are seeing is a page inside a page. The images may not display because of how they are referenced (e.g. absolute or relative) in the HTML code.

The purpose of this iFrame display is to give the person building test scenarios confidence that the correct requests/responses were transacted for this step. The images you don’t see are still going to be requested during the load test, and all of the performance metrics such as response times will be accurate. Virtual users do not render the pages. No one will actually “see” the transactions during a load test because there is no need to see the pages, and our tool is measuring the Throughput, Error Rate, Response Times, Requests per Second, and Concurrent Users from many different load generation machines that process each virtual user going through the steps.

Unfortunately, our load testing tool doesn’t have a feature for what you need. Normally this is a simple process of putting a domain name to IP address mapping in a configuration file on a particular machine. The process is not as easy in our situation because we use dynamically instantiated servers from the Amazon cloud; thus, we do not know what IP addresses we will be using before you run your tests. We would need to create new images specifically for your tests, and we would need to make changes to our core launching software that instantiates the images so that new machines are launched with this particular host mapping. If you would like us to coordinate this custom process, please contact our support. Additional charges per test will apply.

Yes, you may add Storm on Demand users for any load test. This allows you to spike test traffic to a much higher level than your subscription.

For example, if you have a Hurricane subscription for 5,000 maximum concurrent vusers, you can purchase an additional 10,000 Storm on Demand users for a total of 15,000 vusers on a single test. Those 10,000 on demand users would be available for only one load test, and the 5,000 are available for other tests throughout the month.

Yes, and only if you want to. LoadStorm has a checkbox on the scenario edit page to turn download images on or off. There is a similar option for controlling the download and execution of Javascript files.

Normally performance issues arise in the process of generating the dynamic page, and web servers can serve static images very quickly; however, we understand that many of our clients have websites with a large amount of media-related content, and load test results from those sites could be significantly affected by not downloading that content. Image download is especially relevant to performance if bottlenecks are in bandwidth or other network communications.

There is currently no functionality to add gzip to the LoadStorm request header. We hope to add that feature in an upcoming release. “Accept-Encoding: gzip, deflate” cannot be sent in the request generated by LoadStorm.