Building Test Scripts Archives – LoadStorm

Excellent question. Click here for step-by-step instructions to begin using LoadStorm.

Depending on the test requirements, third-party servers should be set to “Ignored” or “Archived”. Requests that would go to ignored or archived servers will be skipped instead. Skipped requests prevent unnecessary traffic to advertisements, analytics, or other third-party services. You may encounter a server already set to ignored as a part of our blacklist. Please contact [email protected] about enabling a blacklisted server for your testing needs.

Ignore a Server

Please note that by ignoring a server, any requests normally sent to that server will not be sent.

1. Click on the Build menu, then on the Server tab on the top.
2. Double-click the server you want to ignore.
3. Click the Ignore button. No requests will be sent to this server.

Archive a Server

Archiving a server works the same as ignoring a server, but it’s a separate category. This way you can separate the servers you may never need to test from servers that you may not want to test at the moment. To reiterate, archiving a server means it is ignored. Any requests normally sent to that server will not be sent.

1. Click on the Build menu, then on the Server tab on the top.
2. Click the server you want to archive to select it.
3. Click the Archive button. No requests will be sent to this server.

Most people create several scripts to accurately reflect different types of users. Scripts execute simultaneously in order to hit your site with the correct mix of traffic. You control the weighting of each script by giving it a number relative to the others. For example, if you have 2 scripts and you give one a weighting of 4 and the other a weighting of 1, then the first will generate 80% of the traffic and the second 20%. This 80/20 could be realistic for a retail e-commerce site where most of the people (80%) are browsing the product catalog, while a minority of people (20%) are going through a buying experience. You can of course throw in a low-weight scenario to represent some internal employees adding products or issuing refunds.

Another way of entering the weight of the traffic is to just enter the exact percentage you would like each script to carry. For the example above you would simply enter 80 and 20. The only thing you have to be very careful about is that the percentages add up to 100. Otherwise, it will work like the above description, which is different.

There is no limit to the number of scripts you can create, and you can use many scripts in a test. We encourage you to build as many as necessary to accurately reflect the real traffic patterns on your site. That will give you the most meaningful metrics for tuning your applications.

We also simulate the user “think time” with random pauses after each page. Real users click on something, then read the page or fill in a form, and then click on something else. Thus, you don’t want to have a simulated virtual user clicking on something every .5 seconds – it would not give you meaningful results if you want to see how your site holds up in the real world.

Also see: Virtual User Calculations about calculating the amount of virtual users needed for a realistic test.

It helps with performance engineering. We recommend to not begin a large test at the maximum number of virtual users. Rather than run a test for 10 minutes beginning with 5,000 and ending with 5,000, we suggest to start smaller, for example, with 500 users and increase to 5,000 over an hour. This approach has the advantage of allowing your system to get all the parts working properly at smaller load (e.g. caches, threads, database connections) before the heavier volume starts exposing potential bottlenecks in your application. Also if the target server begins to fail during the test this provides more granularity into the number of concurrent users where failure begins. If the test starts at 5,000 and fails all that can be learned from the results is that it fails somewhere between 1 and 5,000 which isn’t very useful.

From what we’ve seen in thousands of load tests with LoadStorm, it is common to see useful patterns in the metrics before the peak traffic is reached, and these metrics point to areas of performance limitations that can be tuned.

Load testing is invariably coupled with performance tuning – an iterative process of test/tune/test/tune. Therefore, we recommend you ramp the volume in order to get more value from the test results.

Yes, there are two additional ways for you to white list our traffic:

1. You can customize the user agent for each script which your firewall can see in the request headers of inbound requests. However, if your test scripts contains HTTPS requests, then those headers will be encrypted. Visit our learning center for more information on modifying a script’s default settings such as the User Agent.
2. If you are not testing against your production server, we recommend trying a non-standard port number to identify the traffic as load testing. LoadStorm allows you to use any port number and can be designated for each target server.

Yes. Once you’ve created a script, you may modify form POSTs in order to log into your application with test user data.

We recommend you visit our learning center, where we go over all of the options for parameterization in a script. A brief explanation is given below, but we also offer video tutorials on common examples such as parameterizing a login form.

While creating a recording using developer tools in your browser, log into your application, and save the recording as a HAR file. Then upload the recording into your LoadStorm account. Open your script and select the parameterization tab.

After that, click BUILD in the left navigation and switch to the User Data tab at the top. Here you can upload your CSV which contains necessary login info (username/email, passwords). Note that the first row’s data will be read as column headers and not row data. You can also choose to “generate data” on-the-fly in LoadStorm. If you need to embed a comma inside a particular data field, simply place quotation marks around all field data in the file.

Finally, edit the necessary script and find the POST request that submits the form. Modify the form to include Custom Data with CSV user information. To add the test user data, you must select “custom” under the form modifications options, and select the data from a CSV file.

This simulates traffic more realistically. Instead of simulating one user logging in over and over, many different users log in instead.

Here are some thoughts on increasing RPS in LoadStorm tests:

1. Reducing the think time added after each page in a script; the minimum pause is 5 seconds, so set both min & max to 5 seconds.
2. Increasing the number of concurrent users.
3. Reducing the response time from your server which decreases the time LoadStorm has to wait for a request to complete.
4. Compressing the size of the response is one way to decrease the time it takes for the request to complete as well as save on throughput.

RPS will always be impacted by the types of pages/resources your scripts are requesting. The way it works is that LoadStorm cranks up the starting concurrent users shortly after test starts. It then walks each user through the script assigned to it. The GET requests will be made in asynchronous bursts of up to 6 at a time such as HTML, images, JS, CSS, and XML. Each POST request will be made on its own to ensure that any dynamic response text from earlier requests was available for to be passed into the POST data. This also ensures that any response data from the POST is available for subsequent requests. When those responses are all returned, then the system pauses a random time between the Min & Max “Think Time”, then it requests the next page. This process continues until the end of the script. So, having more users going at the same time, without waiting more than a few hundred ms to get a response, will increase RPS.

Unfortunately, our load testing tool doesn’t have a feature for what you need. Normally this is a simple process of putting a domain name to IP address mapping in a configuration file on a particular machine. The process is not as easy in our situation because we use dynamically instantiated servers from the Amazon cloud; thus, we do not know what IP addresses we will be using before you run your tests. We would need to create new images specifically for your tests, and we would need to make changes to our core launching software that instantiates the images so that new machines are launched with this particular host mapping. If you would like us to coordinate this custom process, please contact our support. Additional charges per test will apply.