Mr. Prickett, my 12th grade English teacher, was one of my all-time favorites. One day while I was arguing with him about why he took off points on my paper, this popped out of my mouth: “I assumed you knew the character because it was part of the assignment!”
He proceeded to walk to the chalk board (I’m old, before whiteboards), and he wrote “A S S U M E” in huge letters. Now he had everyone’s full attention. As he explained, he circled the letters for emphasis, “for U to assume is to make an A S S out of both U and M E!
I’ve forgotten most of what he taught us, but I’ll never forget that lesson. So I decided to not assume that people coming to our sight will know much about load testing. Hence, this post. I hope you find it useful because it isn’t intended to talk down to anyone.
There is too much to say about load testing in just one blog post, so I will break this down into the key questions: what, why, who, when, how, and where. Let’s start with “what”.
What is Load Testing?
Well, we aren’t talking about how much a crane can lift. That’s a different type of load testing. We are dealing with the volume of traffic in web applications.
In laymen’s terms – finding out if the system can handle the number of users expected.
In techspeak – the process of quantifying an application’s response with key performance indicators (KPIs) to an expected volume of simulated concurrent users.
In Wiki – the practice of modeling the expected usage of a software program by simulating multiple users accessing the program concurrently.
Terminology
The term “load testing” is often used synonymously with volume testing, stress testing, and performance testing. Occasionally, you might hear someone call it reliability testing. While I agree that volume testing is the same as load testing, I see performance testing and stress testing as slightly different.
Performance testing involves more monitoring of the internal components of the system, such as the percentage of CPU consumed during the test. Load testing is focused on what the end user sees such as response times or page errors.
Load testing does NOT try to break the system. Nor is it trying to push the app beyond the normal envelope. That’s stress testing. Load testing should have a pre-defined amount of traffic that is acceptable to the product manager or operations leader (or some muckedity muck in charge). Therefore, the goal of load testing is to verify that the web app can function at that level of volume.
When no expectations for usage limits or boundaries exist, then load testing becomes stress testing. Stress testing says, “I have no idea how much volume it can take, so just crank it up until it crashes.” Or something like that.
Performance Tuning
While load testing may crash the system, that’s considered a bad thing. We want (and hope) that our load test will run smoothly and prove that the app and infrastructure can perform at a certain level. No runs, no drips, no errors. Just low response times and lots of fulfilled HTTP requests.
However, if load testing does bring the application to its knees, then it is still a positive outcome as long as the development team uses the test data to improve the system. Performance tuning is the normal activity that would be associated with remediation of a web app that cannot handle the expected load. Tuning can involve many adjustments to the system.
I’ll admit that my first thoughts when a web app fails under load are something like, “Let’s buy a bigger server.” That’s not a great way to tackle performance unless you have plenty of money and no system architect or developer resources.
While hardware may be an issue, many other approaches should be tried as well. A few examples are: optimizing the application code, modifying caching settings, and load balancing. Each of these examples are worthy of several blog posts, but we will not delve into those subjects now.
What is a load testing tool?
Load testing tools create large volumes of traffic against your website or web application. These tools simulate the actions normally taken by real users. In most cases, the tool will generate large numbers of http requests against the target application (server).
These requests are configured in a proportion that mimics the behavior of actual users so as to get a realistic traffic pattern in the load test. For instance, an e-commerce site may have 80% of traffic comprised of people browsing through the product catalog, while another 15% are putting things in a shopping cart, and the final 5% are putting in their credit card info to complete the purchase. The load testing tool will simulate each of these actions in the proper percentages.
Most tools are software you install on your server. A few new tools are offered as on-demand, hosted web applications.
One of the key capabilities of a good load testing tool is scalability. You need high volumes of virtual users and the ability to ramp up and down quickly without much effort on your part.
Load testing tools also provide reporting to show the virtual users’ experiences at the test site. Such metrics as page response times and error rates are captured throughout the load test and presented to the tester in tables and graphs. This data is critical to the testers and developers because it tells them how the system handled the volume from the user’s perspective.
A few years ago, Amplitude Research conducted a study on web application load testing. Several levels of employees including programmers, project managers, product managers, and executives were surveyed on various issues related to web app performance and testing. The results showed strong support for load testing across the board. Most preferred to outsource the testing. One of the top priorities stated was the ability to handle complex scenarios to simulate all types of user experiences. Easy to understand reporting was also a top priority.
The respondents indicated that price was not a significant factor in their satisfaction with the load testing. Yeah…right.
In the next installment of Load Testing Fundamentals, we shall tackle the concept of, “Why Load Test?” I hope this was helpful.