Load Testing Saves Money and Time – LoadStorm

If you want to lower the cost of your software products and speed up your development time, then invest more resources in quality assurance from the beginning of your project.

When I first encountered that concept about a decade ago, it caught me off guard. I had a similar reaction to when a backpacking book stated, “If your feet are cold, put on a hat.”

The county where I live gets 300+ inches of snow in the winter. Dealing with the cold is a priority. Putting on a hat to get my feet warm was NOT intuitive. But the facts bear it out because 80% of our body heat is lost through our head; thus, a warm hat keeps the core warmer and allows blood to flow freely to our extremities. Hmmm.

Likewise, it is NOT intuitive that spending more time on QA activities will reduce the time to develop and release a software product. Again the facts bear it out.

This chart shows the relationship between defect rate and development time (and therefore cost). I got the graph and data from Steve McConnell’s book “Rapid Development”

McConnell makes the sound argument that, “…higher quality (in the form of lower defect rates) and reduced development time go hand in hand.” I agree. And time translates very clearly to money.

That’s not the reality I’ve lived as a developer. Most of my project managers have invariably found our team behind schedule against a delivery deadline (set by the PM), so they saved time by cutting back on QA.

Does the following sound familiar?

• “We don’t have time for code reviews.”
• “Just unit test the best you can. We will catch more bugs in system testing.”
• “There isn’t time for load testing. Performance testing and tuning will have to wait until somebody starts complaining.”

Those statements were my development reality…until I met Roger Campbell. He’s the one that bought me Rapid Development. Roger educated me on the basics of how it is cheaper and faster to avoid software bugs than it is to fix them later.

Capers Jones wrote or several books and articles in the 1990’s that provide excellent data supporting the notion that QA practices reduce product costs and development time (e.g. Applied Software Measurement: Assuring Productivity and Quality, Assessment and Control of Software Risks, Determining Software Schedules, etc.)

Jones concludes that poor quality is the most common reason for overruns and is a top reason for canceled projects. He demonstrates that approximately 95% of pre-release defect removal has the optimal impact on the cost of software (least effort, highest user satisfaction, quickest release schedule). Therefore, if greater than 5 percent of your bugs are found after a code release, then your development time and costs are likely to be sub-optimal. It’s cheaper and faster to perform QA throughout the product development cycle – including requirements review right upfront.

I submit that performance testing is a critical part of the QA process. Further, I strongly believe that performance testing and load testing will save time and money in the overall project.

There are several types of defects that can only be found through forms of load testing. A good example is a memory leak that can be detected with a long soak test. Similarly, endurance testing or stress testing can uncover defects such as failure to close connections or database cursors. Endurance testing, a.k.a. soak testing, puts a high volume of simulated users on a web application over an extended period of time. The objective is to discover system behavior over significant duration and load.

When a development team (including project and/or product manager) decides not to give proper attention to bug detection, they are effectively deciding to shift increased costs to the back-end of a project. That is irrational.

Stakeholders for the software product should understand the ramifications of poor QA. Budgets are tight. Time is at a premium. It seems logical to trim testing – especially load testing. On the surface it would seem that testing response times and user volume can wait.

It is precisely BECAUSE budgets are tight and time is at a premium that QA must be adequately addressed from the very earliest planning of the project. That is how you save money and time. Performance, stress, endurance, spike, and general load testing absolutely are contributors to lower costs and expedited delivery of working code. Perhaps most importantly in the long term, quality assurance practices improve customer satisfaction.