Load testing is simulating a high volume of concurrent users against a software application.The primary objective? To measure how well the application performs during heavy traffic.
In terms of web applications, a load test hits the site with http traffic in a realistic fashion that mimics activities by users such as exploring a product catalog, logging in, and posting comments to blogs. The goal is to use the resulting test metrics to find slow performing aspects of your site in order to make coding changes that will improve the efficiency of your web application.
Tools are almost always employed to create hundreds or thousands of virtual users taking actions on your site. The virtual users move through your site clicking on links, logging in, posting comments, browsing your product catalog, and buying coffee mugs with a credit card.
Load testing tools capture metrics such as response times experienced by the end user. If a tester finds that a login process takes 2 seconds with 5 users, but the same process takes 30 seconds with 50 users, then the conclusion is that the web application has scalability problems. Performance tuning is called for.
A website can quickly get hammered with traffic. For instance, the Slashdot effect is notorious for generating a very large volume of clicks to your site in a short period of time. Determining the performance of your web application under load is extremely valuable. Slow performing sites have been shown to lose revenue because users won’t wait. Web performance tuning can increase your profit by up to 12%.
The 4 KEY COMPONENTS of Load Testing
- Understanding the environment of the application being tested
- A test plan defining user behavior
- Tools to generate load and capture metrics
- Analysis of performance indicators
Additional explanation and detail is offered below for each of these four key components of load testing.
Understanding the Environment of the Application Being Load Tested
- Figure out the architecture of physical layers and logical layers
- Study any system documentation that shows the big picture, not the coding
- Ask plenty of questions of project stakeholders to see how the system will be used
- Determine expectations of IT leaders, executives, and especially the head of marketing
Some people call this the “context” of the load test. A good way to understand the context or specific situation around your load test is to play detective. Review a list of load testing questions and analyze them for which ones will be appropriate for your project. Don’t be afraid to ask. Above all – listen carefully to the answers!
A Load Test Plan Defining User Scenarios
Your test plan should contain:
- User types – administrators, buyers, window shoppers, bloggers, commenters, etc.
- User success criteria – if they wait over 5 seconds will they leave?
- Business success metrics – examples are: concurrently process 5,000 anonymous users shopping, 500 processing their credit card, 200 posting comments about the products, and 50 clerks updating the product catalog
- System success criteria – examples are: 98% of all responses time to last byte under 3 seconds, memory utilization under 60%, CPU utilization averages under 35% and never reaches 85%, etc.
- Test scenarios – for each user type, describe a series of actions that they can take
- Volumes – for instance: ramp up traffic from 100 to 5,000 concurrent users over 60 minutes, simulating 80% of visitors as anonymous, 10% shopping cart experience, 10% authenticated users performing CMS transactions
Tools to Generate Load and Capture Metrics
There are dozens of load testing tools, and a few of these will fit your needs. A thorough list is available at performance testing resource and linking site. Some tools have been around for 20 years, have the most features, are the most expensive, have long learning curves, and tend to be for companies that have a large budget with a dedicated testing team. Open source tools are plentiful and are free to download, but require you to provide the hardware and usually require significant tech skills to configure. New cloud tools abstract the hardware from you, allowing you to focus on the other aspects of running the tool, and their cost is based on usage rather than license fee. Choosing the best for you will ultimately come down to a balance between ease/features, and the price your project can afford.
Representatives of each category:
- LoadRunner – very expensive, takes months to learn, extensive scripting capability using C language
- JMeter – Desktop tool, free, moderate learning curve, extensible by Java coders
- LoadStorm – load testing in the EC2 cloud, inexpensive, short learning curve, limited scripting capability
Analysis of Performance Indicators
The purpose of load testing is to apply large volumes of virtual users to the web application or site to determine how it responds to the heavy traffic. Many confuse load testing with analyzing page performance. It is a good idea to examine the pages of your site or app to see how long it takes for each resource (html, images, stylesheets, Javascript, XML) to load. There are aspects of rendering that you should consider as part of the user experience. However, the big difference is that you analyze page performance with one user at a time, and in load testing you have hundreds or thousands of users all hitting the site at the same time. Let’s start with load testing metrics:
- Response times – milliseconds it takes to receive a response to a certain request made
- Time to first byte (TTFB) – the timer starts when the request is sent, and the timer stops immediately when a response header is first registered
- Time to last byte (TTLB) – the timer starts when the request is sent, and the timer stops when everything in the response payload is received
- Requests per second – the average number of requests being received by the application’s front end, regardless if they get a response
- Throughput – the amount of data flowing in/out of the web site, measured in kilobytes per second
- Error rate – which errors (if any) are occurring as the traffic increases, measured as a percentage of total requests
Load Testing Summary
Load testing involves applying traffic to a system and measuring how it performs. Tools simulate heavy volumes of users taking various realistic actions in the system. Web applications are especially more deserving of an investment in load testing because large amounts of traffic can hit a site in a short amount of time. The stories of website crashes under load are common, and an outage is expensive.
The objective of load testing can simply be to confirm that your website can handle a certain number of concurrent users. For instance, it is common for a publishing company to release a new book and receive 5,000+ users within the first hour of official press releases. The load test can be run a few weeks before the book release with 5,000 concurrent users, and the test results may show that the purchasing process bogs down to an unacceptably slow average response time at 1,000 concurrent users. If the load test objectives are to support 5,000 concurrent users, then this system failed the test. The load testing was successful, but the web application performance was not successful.