Load Testing versus Performance Testing
Performance Testing = how fast is the system?
Load Testing = how much volume can the system process?
Performance testing seems to me to be much more broad than load testing. Consider:
- A web developer can test the speed at which a page renders in a browser, and that is testing performance. Yet, that test would have nothing to do with load.
- I might analyze the efficiency at which my database processes a single specific SQL query, and the resulting speed of delivery of the records can be the slowest component of the whole page building process. Measuring that speed is about performance, but only one transaction is involved (small load).
- Load testing is usually focused on metrics like requests per second and concurrent users (the cause); whereas performance testing is more concerned with response times (the effect).
Testing for performance, part 1: Assess the problem space
Imagine you’re a competent programmer who has worked with a number of different clients in many different technologies and who has just been assigned to your first project as a performance tester. To complicate matters, the technology involved happens to be fairly complicated and unfamiliar to you; this is not a project similar to one in your portfolio.
Testing for performance, part 2: Build out the test assets
Stage the environments, identify data, building out the scripts and calibrate your tests
As we look at the various environments, tools, data and scripts we use to build our tests, we will try to tie those artifacts back to the work that we did in the first article when we assessed the problem space. Often, as we build out the test assets, our understanding of the problem space changes, becoming more (or sometimes less) detailed, and often we are required to reconsider some of the decisions we made before we got down into the details.
Testing for performance, part 3: Provide information
In this article we look at some of the last heuristics: execute, analyze, and report:
- Execute: Continually validate the tests and environment, running new tests and archiving all of the data associated with test execution
- Analyze: Look at test results and collected data to determine requirement compliance, track trends, detect bottlenecks, or evaluate the effectiveness of tuning efforts
- Report: Provide clear and intuitive reports for the intended audience so that critical performance issues get resolved
Performance Testing according to Goranka Bjedov, Senior Test Engineer on Google Test Blog
Think of performance testing as another tool in your testing arsenal – something you will do when you need to. It explores several system qualities, that can be simplified to:
- Speed – does the system respond quickly enough
- Capacity – is the infrastructure sized adequately
- Scalability – can the system grow to handle future volumes
- Stability – does the system behave correctly under load
Simple & Effective Performance Tips
Your end users complain that your Web pages load slowly and they don’t intend to wait! So you need to dust off your system topology diagram and understand which servers user traffic travels. Then you need to determine if CPUs are busy on these systems, if they are busy (i.e., above 85%), you need to track down which process is using the CPU cycles; if CPU seems fine (i.e., under 85%), then ask yourself if memory is running out? If your system is memory-bound, you could be paging to disks – disk I/O is a performance killer. If memory is the issue, find out which process(es) are hogging memory – and they could be abusing memory too (e.g., memory leak, or loading large Word files into memory for processing); if your systems have pletty of free memory, then you need to check if your disk I/O is high. If disk I/O is high, you need to find out which process or thread is writing to and reading from the disk! Finally, if all the above checked out fine, ask yourself, if network bandwidth is maxed out: traffic is so high that responses from your Web site simply cannot push through the thin network “pipe”. But ask yourself, is your application supposed to push that much data?
Identify common website performance bottlenecks:
- Source (what component they occur on)
- Symptom (how you know there’s a problem)
- Causes (what creates the problem)
- Measurements (how to nail it)
- Cures (how to make it go away)
There are several good posts about why people should tune their apps. Lots of other good places to find info on HOW to tune. Web performance tuning does a great job of using case studies and research by the big companies to prove the correlation between web performance and profit. It’s obvious – slow pages make users go away; thus, revenue is lost.
Top Ten Web Performance Tuning Tips by Patrick Killelea, author of Web Performance Tuning is a good read for ideas. Patrick rightly points out that too many times performance testing is overlooked.
Java Performance Tuning Tips has a large number of tips categorized by topics such as JDBC, JMS, EJB, Application Servers, J2EE, J2ME, etc.
Apache Performance Tuning has suggestions on improving performance of the web server. It provides advice on run-time configuration issues, avoidance of swapping, and compile-time configuration issues.
Ensuring Web Site Performance – Why, What and How to Measure Automated and Accurately
Difference in perf and load testing – part 1
Difference in perf and load testing – part 2
FAQ – Performance & Load Testing on Software QA Forums
Performance testing: Does your Web site make the grade? on TechRepublic
How to Test and Improve Website Performance by Bob Rankin
How to Make Developers Write Performance Tests
Nuts and bolts of performance testing by Abhinav Vaid on StickyMinds.com
Web performance test using Visual Studio
Performance Testing In Agile Environments by HP
Rapid Bottleneck Identification – A Better Way to do Load Testing by Oracle
Serverside.com looks at performance testing of Java apps
Types of performance testing by TestingGeek.com