A web developer called me the other day and asked, “Is load testing the same as performance testing?”
I pondered for a moment and replied, “Not really.” Hmmmm…now I needed to support my answer. Thus began a hunt for good answers to my client’s question.
Searching Google, gleaning from our interviews, and asking respected colleagues were parts of my pursuit for the truth about load vs. perf. Sorry to abbreviate, but I’ve heard cool engineers shorten it to “perf”, and I desperately want to be cool. (in a geeky way)
The answer isn’t black and white. However, I have heard software testing professionals say load testing is more aligned with black box testing and performance testing with white box testing. Thus, performance involves getting inside the various components of a system.
Strong opinions are expressed by many experienced testers – and the opinions vary. In this post, I will give my perspective and share quotes from people smarter than myself about the topic. Herein lies the results of my hunt.
Personally, I believe load testing is a subset of performance testing. My point of view will be relative to web applications or websites. In the same way web developers usually will view testing differently from QA Managers, non-web software testers may not agree with my narrow definitions of perf vs load.
My Simplified Definitions
Performance Testing = how fast is the system?
Load Testing = how much volume can the system process?
Load Testing = how much volume can the system process?
Performance testing seems to me to be much more broad than load testing. Consider:
- A web developer can test the speed at which a page renders in a browser, and that is testing performance. Yet, that test would have nothing to do with load.
- I might analyze the efficiency at which my database processes a single specific SQL query, and the resulting speed of delivery of the records can be the slowest component of the whole page building process. Measuring that speed is about performance, but only one transaction is involved (small load).
- Load testing is usually focused on metrics like requests per second and concurrent users (the cause); whereas performance testing is more concerned with response times (the effect).
How Our Industry Defines Them
Wikipedia has this to say about load testing:
“Load testing is the process of putting demand on a system or device and measuring its response.
When the load placed on the system is raised beyond normal usage patterns, in order to test the system’s response at unusually high or peak loads, it is known as stress testing. The load is usually so great that error conditions are the expected result, although no clear boundary exists when an activity ceases to be a load test and becomes a stress test.
There is little agreement on what the specific goals of load testing are. The term is often used synonymously with software performance testing, reliability testing, and volume testing.”
Note that last sentence – synonymous with performance testing. Yes, I agree. They are often used interchangeably. Hence the need to more clearly define them.
Wikipedia defines performance testing as:
“In the computer industry, software performance testing is used to determine the speed or effectiveness of a computer, network, software program or device. This process can involve quantitative tests done in a lab, such as measuring the response time or the number of MIPS (millions of instructions per second) at which a system functions. Qualitative attributes such as reliability, scalability and interoperability may also be evaluated. Performance testing is often done in conjunction with stress testing.”
Answers.com provides this definition for load testing:
“Load testing is the process of putting demand on a system or device and measuring its response. There is little agreement on what the specific goals of load testing are. The term is often used synonymously with software performance testing, reliability testing, and volume testing.”
Answers.com provides this definition for performance testing:
“In software engineering, performance testing is testing that is performed, from one perspective, to determine how fast some aspect of a system performs under a particular workload. It can also serve to validate and verify other quality attributes of the system, such as scalability, reliability and resource usage.”
SearchSoftwareQuality defines load testing this way:
“Load testing is the process of subjecting a computer, peripheral, server, network or application to a work level approaching the limits of its specifications. Load testing is a part of a more general process known as performance testing.”
SearchSoftwareQuality defines performance testing this way:
“Performance testing is the process of determining the speed or effectiveness of a computer, network, software program or device. Performance testing can also be used as a diagnostic aid in locating communications bottlenecks.”
Friends and Colleagues Speak Out
Now let’s hear from the experts. There have been several web developers and testers that have graced us with their insights via interviews. Those are in our blog, so I plucked out their thoughts on our perf vs. load question.
Hunting With Google & Twitter
- Load Testing
- Stress Testing
- Volume Testing
Web load testing is:
- similar to, but not synonymous with performance testing
- concerned with the volume of traffic your website (or application) can handle
- not intended to break the system
- viewing the system from the user perspective
- associated with black box testing
Web performance testing is:
- a superset of load testing
- concerned with speed and efficiency of various components of the web application
- useful with only one user and/or one transaction
- viewing the system from the architecture perspective (behind the server side curtain)
- associated with white box testing