Building Test Scenarios Archives – LoadStorm

This is a very common question. In fact, a good example is the email this week from a new user stating, “Just tried to test your service and it came back with these comments:

I am not sure what to do now…Why do I have to verify these servers?”

Are You Load Testing Facebook?

However, the error messages above indicate that his test plan will be hitting servers that aren’t verified (authorized for testing). Put another way, his scenarios contain requests that go to servers that aren’t his, and LoadStorm gets in trouble when our customers load test other people’s servers without their permission.

Does he really want to hit Twitter and Facebook for his load test? If so, is he testing their ability to handle his load? If so, does he have their permission to hammer on their servers? Probably not. They tend to get upset when someone utilizes a cloud load testing tool to simulate thousands of VUsers against their social media platform. Please read the terms and conditions of use for those services.

And where do those VUsers originate? LoadStorm. So who gets the phone call from their lawyers? We do.

Prevent Denial of Service Attacks

Our verification process is designed to prevent LoadStorm from being used as a Denial of Service (DOS) weapon. By placing a code on your home page that LoadStorm can request and confirm, our system confirms that you have the control over that server; thus, it is safe for our tool to hammer it with VUsers.

Recommendation

To verify a server follow the instructions in our learning centers for LITE or for PRO.

Ignore all third-party servers that are not crucial to your web application. LoadStorm will not make any requests to ignored servers. Remember the goal is to test the scalability of your web servers, and not those of third-parties. If you feel there is a third-party server that is crucial to your testing needs please email them and copy [email protected] requesting permission from their company to let you use our software to load test their web servers and yours as part of your testing needs.

Most people create several scripts to accurately reflect different types of users. Scripts execute simultaneously in order to hit your site with the correct mix of traffic. You control the weighting of each script by giving it a number relative to the others. For example, if you have 2 scripts and you give one a weighting of 4 and the other a weighting of 1, then the first will generate 80% of the traffic and the second 20%. This 80/20 could be realistic for a retail e-commerce site where most of the people (80%) are browsing the product catalog, while a minority of people (20%) are going through a buying experience. You can of course throw in a low-weight scenario to represent some internal employees adding products or issuing refunds.

Another way of entering the weight of the traffic is to just enter the exact percentage you would like each script to carry. For the example about you would simply enter 80 and 20. The only thing you have to be very careful about is that the percentages add up to 100. Otherwise, it will work like the above description, which is different.

There is no limit to the number of scripts you can create in a test plan. We encourage you to build as many as necessary to accurately reflect the real traffic patterns on your site. That will give you the most meaningful metrics for tuning your applications.

We also simulate the user “think time” with random pauses between pages. Real users click on something, then read the page or fill in a form, and then click on something else. Thus, you don’t want to have a simulated virtual user clicking on something every .5 seconds – it would not give you meaningful results if you want to see how your site holds up in the real world.

Also see: Virtual User Calculations about calculating the amount of virtual users you need for a realistic test.

It helps you with performance engineering. We recommend that you do not begin a large test at the maximum number of virtual users. Rather than run a test for 10 minutes beginning with 5,000 and ending with 5,000, we suggest that you perhaps start with 500 users and increase to 5,000 over an hour. This approach has the advantage of allowing your system to get all the parts working properly at smaller load (e.g. caches, threads, database connections) before the heavier volume starts exposing potential bottlenecks in your application.

From what we’ve seen in thousands of load tests with LoadStorm, it is common to see useful patterns in the metrics before the peak traffic is reached, and these metrics point to areas of performance limitations that can be tuned.

Load testing is invariably coupled with performance tuning – an iterative process of test/tune/test/tune. Therefore, we recommend you ramp the volume in order to get more value from the test results.

Yes, there are two additional ways for you to white list our traffic:

1. The user agent in the HTTP header includes LoadStorm. However, if your test scripts contain steps with HTTPS requests, then those headers will be encrypted.
2. If you are not testing against your production server, we recommend trying a non-standard port number to identify the traffic as load testing. LoadStorm allows you to use any port number and can be designated for each target server.

Yes. Once you’ve created a script, you may modify forms in order to log into your application with test user data.

We recommend you visit our learning center, where we go over all of the options for parameterization in a script. A brief explanation is given below, but we also offer video tutorials on common examples such as parameterizing a login form.

While creating a recording using developer tools in your browser, log in to your application. Save the recording as a HAR file and upload it into your account. Open your script and select the parameterization tab.

After that, upload your CSV which contains necessary login info (username/email, passwords) into the Build… User Data section of LoadStorm. Note that the first row’s data will be read as column headers and not row data. You can also choose to “generate data” on-the-fly in LoadStorm. If you need to embed a comma inside a particular data field, simply place quotation marks around all field data in the file.

Finally, edit the necessary script and find the request that submits the form. Modify the form to include test user information. To add the test user data, you must select “custom” under the form modifications options, and select the data from a CSV file.

This simulates traffic more realistically. Instead of simulating one user logging in over and over, many different users log in instead.

Here are some thoughts on increasing RPS in LoadStorm tests:

1. Reducing think time between pages in a script – minimum is 5 second pause, so set both min & max to 5.
2. Increase the number of concurrent users.
3. Reduce the response time from your server which decreases LoadStorm’s waiting for a response.

RPS will always be impacted by the types of pages/resources your scripts are requesting. The way it works is that LoadStorm cranks up the starting concurrent users shortly after test starts. It then walks each user through the script assigned to it. One step is executed (such as home page), and all the images, JS, CSS, XML are requested separately. When those responses are all returned, then the system pauses a random time between the Min & Max “Think Time”, then it requests the next page. That continues until the end of the script, whereupon that virtual user is retired. After a few seconds, another vuser is created and the process starts again. So, having more users going at the same time, without waiting more than a few hundred ms to get a response, will increase RPS.

Unfortunately, our load testing tool doesn’t have a feature for what you need. Normally this is a simple process of putting a domain name to IP address mapping in a configuration file on a particular machine. The process is not as easy in our situation because we use dynamically instantiated servers from the Amazon cloud; thus, we do not know what IP addresses we will be using before you run your tests. We would need to create new images specifically for your tests, and we would need to make changes to our core launching software that instantiates the images so that new machines are launched with this particular host mapping. If you would like us to coordinate this custom process, please contact our support. Additional charges per test will apply.

Yes, you may add Storm on Demand users for any load test. This allows you to spike test traffic to a much higher level than your subscription.

For example, if you have a Hurricane subscription for 5,000 maximum concurrent vusers, you can purchase an additional 10,000 Storm on Demand users for a total of 15,000 vusers on a single test. Those 10,000 on demand users would be available for only one load test, and the 5,000 are available for other tests throughout the month.

There is currently no functionality to add gzip to the LoadStorm request header. We hope to add that feature in an upcoming release. “Accept-Encoding: gzip, deflate” cannot be sent in the request generated by LoadStorm.