I’ve known some web Product Managers that require at least one 72 hour load test on a monthly basis. In fact, a couple are customers of LoadStorm.
The reasoning behind conducting such long tests is that memory leaks can be very difficult to find in a 1 hour test and should be detectable after 72 hours. Comparisons of memory consumption in the first hour to the 72nd hour under the same load may expose any defects in the code that leave memory wasted.
I call this endurance testing. Many developers refer to it as soak testing or reliability testing. The goals for success of your load testing would be better to include at least one long duration test because it checks the reliability in a way that cannot be tested in any other way.
Wikipedia defines soak testing as:
Soak testing involves testing a system with a significant load extended over a significant period of time, to discover how the system behaves under sustained use.
Mike Kelly at SearchSoftwareQuality believes endurance testing and soak testing are slightly different:
It’s similar to endurance testing and burn-in testing, and is often used interchangeably. For me, the key distinction to soak testing is that load remains high throughout the run, while with an endurance test my load may vary based on the usage model I’m executing.
For my purpose, I’ll use these terms interchangeably. The only difference seems to the volume that Mike uses. I view the duration as the significant factor that makes a soak test.
It’s common to overlook endurance testing simply out of ignorance of its importance. Even worse, programmers may think too highly of their skills to consider memory leaks a possiblity. What is probably the most significant reason for skipping soak testing? Budget. Running a load test for a day or two consumes resources; thus, it’s viewed as expensive. To justify it, calculate the cost of shutting down the system to recover from a failure. It might be easy to get your stakeholders to agree to include endurance testing in your success criteria when they factor in the downtime.
How long can your web application process heavy load without errors or failures? Best to find out now rather than later.