Yesterday, we examined how the new availability of elastic cloud computing has changed load and stress testing for the better. It lowers the cost, increases scalability, and facilitates running tests more efficiently. An article on load-testing.org about load and stress testing got me thinking about how our paradigms in this industry have changed and continue to evolve.
Today, we look at the reasons for why we would want to run tests more frequently and earlier in the software product lifecycle. To borrow a Chicago axiom about voting – test early and test often. It’s the new paradigm.
Wide-spread Adoption of Agile
One of the most important shifts in paradigms in the web application dev industry is the wide-spread adoption of Agile.
Agile software development is a group of software development methodologies based on iterative and incremental development, where requirements and solutions evolve through collaboration between self-organizing, cross-functional teams. The Agile Manifesto[1] introduced the term in 2001.
Notice that I didn’t say, “…the invention of Agile”. I said, “…the wide-spread adoption of Agile”. The idea of short dev cycles was something I’ve been practicing since building dBase III apps for General Motors (is it Government Motors now?). I always loved the ability to code something and show new features to the user within hours. They thought I was really smart! Being Agile in the 80’s really helped my self-esteem and probably got me a few pay increases.
The book Rapid Development by Steve McConnell released in 1996 was very influential on me. It’s emphasis on the quick iterations of coding and testing inspired many of the best programmers I know. The best practices were outstanding because they gave tangible examples of how to implement improved teamwork around what are now a core part of the Agile movement.
The Impact on Load, Stress, and Performance Testing
So what does Agile and Rapid Development have to do with load testing? Well, if you adhere to the principles of Agile, then you will have the mentality that quality should be integral to everything you do. Testing is not something performed as an afterthought by a separate QA team. Load and stress testing is incorporated into the project plan at the very beginning, and performance is given such high priority that it is expected to be tested throughout the development cycle.
The argument can be made that performance testing should wait until all the functional testing is complete and all the bugs fixed. I know plenty of Project Managers and CTOs that believe this myth. Their thinking is something like this, “Why should I care about performance metrics now when I know that we have lots of bugs left in the code?” My answer is twofold:
- Software is never bug-free
- Performance testing early will uncover more defects
- Defects found earlier in the dev cycle are cheaper and easier to fix
- Performance bottlenecks that are profound can easily be caught early regardless of other bugs
- Stress failures in the system architecture are usually much more difficult to find their causes when the application has grown complex
Conclusion
Load and performance testing is different now than it was even 5 years ago. Many of the tools are still popular because of their large install base, but the way stress testing is conducted has been up-ended by improved technologies and methodologies.
If software defects can be found and fixed at a lower cost and with less effort nearer to the beginning of a project, then it is advantageous for all of us web developers to make it an imperative to hunt those bugs all the time – not just after we have coded all the modules. This savings applies to web performance of an entire system as well. I can find an expensive query early on in the life of an app, and I can make sure not to duplicate that mistake in many more areas of the code. Problems are prevented.
I can find poorly configured JVM garbage collection settings before they are disguised by other server connection or threadpool issues. Performance of the system is greatly increased, and some poor programmer on call doesn’t spend 36 straight hours pouring through every bit of source code and network diagrams trying to find the root cause.
Paying attention to performance in every sprint is a good thing. It will save your company money and your team time. Stress testing important to Agile, and the new cloud load testing tools make it much cheaper and easier to run tests over and over and over throughout the software product lifecycle.
The new paradigm is better. It makes everybody involved happier – even the marketing department!