Only LoadStorm 1.5 Archives – LoadStorm

LoadStorm follows links when the session ID is appended to the URL query string because it searches for the text between the open and close anchor tags. If it doesn’t find that text, then the fall back is to match the XPATH. We stored the XPATH when the scenario was created (essentially an address inside the page’s DOM). The XPATH is a unique identifier that allows us to click on the anchor such as an image or the href attribute or execute the Javascript.

Average Duration is the time taken for ALL steps to complete (see step duration calculation), plus the average pause times between steps. Thus, it is the mean time for a specific vuser’s “life”. The actual duration fluctuates with the amount of pause assigned to that vuser. The Average Duration is shown above the steps on the Scenario page.

Duration is the time taken for a scenario step to complete, which is the sum of time to last byte for each request in the step (including HTML, JS, CSS, images, etc.). The Average Duration shown above the steps on the Scenario page is the total of all step durations plus the average pause times between steps.

The Form Data Set field that you see on the Scenario settings page is a mechanism to select an input source for form submission. Using the Form Data Sets functionality, it is possible to submit forms such as a login form and fill-in the fields (e.g. username, password) with data from your system. Other common types of form submission are search terms, credit card numbers, and profile preferences. Any type of form field that can be filled in with text can utilize the Form Data Sets.

You may upload your own data to be used for form submission, or you may use one of our built-in form data sets.

The Tiny, Small, and Huge form data sets are built-into LoadStorm. The only real difference between the three files are the number of rows. Tiny only has 10 records. Small has 100, and Huge has 10,000 records of user information. They contain user information:

• First Name
• Last Name
• Username
• Password
• Email address

Many of our customers use one of our built-in form data sets for registration on their web application or for logging in virtual users.

If you are going to upload your own data into LoadStorm, here is an example of the process for getting user credentials from your system to LoadStorm:

1. Export a CSV file of user information from your user table(s).
2. The first row of the CSV must contain the names of the columns, and these column names will be mapped to the field name in the step that submits the form.
3. The CSV file is uploaded by clicking on the UPLOAD DATA button on the Build tab. This button will be near the bottom, under the Form Data Sets section of the page.
4. You must give the file a label so that you can choose that file to be used at the scenario level.

Once you have the CSV data in LoadStorm (you should see it on the Build tab under the heading Form Data Sets), then you need to edit the scenario and choose the appropriate data set based on the label you assigned when uploading it. This essentially connects the CSV file to your scenario. Then when you add a step for form submission, you will be able to map the fields in the form to the columns of data in the Form Data Set.

The LoadStorm Tour has a video tutorial showing how to create a login form submission. Watch the video on YouTube

When adding a step in LoadStorm, our emulator makes the request to your web application and receives the response containing the HTML code for the page. Our tool then parses the HTML to determine what additional resources (e.g. images, Javascript libraries) need to be requested for that page.

Each of those resources are requested and received. LoadStorm saves all of the HTML and other resources in our database along with other information it needs to reproduce the sequence of events for this step during a load test. The visual rendering you see at the bottom of the step is an iFrame where our tool places a copy of what was received during the step creation process. It is NOT an accurate rendering as you would see it in a full browser without all of the other LoadStorm code and process
“wrapped around” it. Essentially, what you are seeing is a page inside a page. The images may not display because of how they are referenced (e.g. absolute or relative) in the HTML code.

The purpose of this iFrame display is to give the person building test scenarios confidence that the correct requests/responses were transacted for this step. The images you don’t see are still going to be requested during the load test, and all of the performance metrics such as response times will be accurate. Virtual users do not render the pages. No one will actually “see” the transactions during a load test because there is no need to see the pages, and our tool is measuring the Throughput, Error Rate, Response Times, Requests per Second, and Concurrent Users from many different load generation machines that process each virtual user going through the steps.

You can press the “STOP NOW” button at any time. It is found on the Run tab, and the button will only appear during the execution of a load test.

Also, LoadStorm will automatically stop a load test if it can determine that your server has been overloaded. The thresholds for stopping a test are currently when the Error Rate exceeds 50% or the Average Response Time exceeds 15 seconds for the most recently summarized test period.

Important Note for Small Tests: The sample interval is not the entire test but just the most recent part of it, and there must be a minimum number of requests in the sample. This interval is significant because in very small tests the number of failed requests AND the total requests may be few. For example, you must have some requests for at least 2 intervals, which means you have summarized more than 1 minute of requests.

If one of your web pages contains a reference to an image that is not available on your system, then a response header will be sent to LoadStorm have Status Code = 404. Additionally, your web server will send an HTML page indicating the 404 “Not Found” page to alert the user. That’s standard practice in web applications.

The reason LoadStorm reports the 404 errors as HTML is because your web server is sending our system the Content Type = “HTML” attribute. LoadStorm separates the data in the HTML and Other rows by the Content Type.

Yes, and only if you want to. LoadStorm has a checkbox on the scenario edit page to turn download images on or off. There is a similar option for controlling the download and execution of Javascript files.

Normally performance issues arise in the process of generating the dynamic page, and web servers can serve static images very quickly; however, we understand that many of our clients have websites with a large amount of media-related content, and load test results from those sites could be significantly affected by not downloading that content. Image download is especially relevant to performance if bottlenecks are in bandwidth or other network communications.

Page views is a good term when reviewing Google Analytics data for traffic analysis. However, when reviewing load test data, page views can be a confusing metric. We do not track page views; rather we track requests and responses. Let’s look at the difference and how LoadStorm measures interactions with the server.

The page views per virtual user should stay consistent as long as the response time is consistent and no errors. In some tests, the response time and errors are significant and increase over the duration of the test. For example, we see some load tests that have a large number of timeouts (no response to the request in 35 seconds). That means that LoadStorm cannot move to the next request for a virtual user until the response is received or we timeout the request. This drastically changes the equation you may be using to calculate the number of pageviews. Specifically, in one test there were 143,553 errors out of 1,345,534 total requests during a 30 minute test. This failure rate greatly decreased the number of requests LoadStorm was sending during the test.

Also, if a request is sent for a step, LoadStorm is using the response header to capture the content type. A normal HTML page in a test returns, “text/html; charset=UTF-8″. When no response is received, LoadStorm assigns the response content type as, “NONE”. This affects the summary report table because the NONE responses are lumped into the OTHER category rather than the HTML category. Therefore, many of the 143k errors are shown in the OTHER category – not in the HTML category. That means that many of the HTML requests are not even shown in the table as HTML (or page views).

The bottom line is that it’s mathematically invalid to expect linear correlation between page views, requests/responses, and vusers if the target server is not responding well. Everyone has a different definition of test success and failure. Some believe that greater than 1% error rate is a reason to stop the test. Others believe that an average response time greater than 3 seconds is a failure. Regardless how you define failure, timeouts and other server errors degrade the value of your load test metrics.