LoadStorm™

I have customers frequently ask me why LoadStorm has a minimum pause time. They want the load test to click from step to step in less than a second. My reply is that it is unrealistic to have actions of users like that.

People have to think about what they see, read it, make a decision, then click. Each person will respond to the delivered page in a slightly different time frame. Think time is variable because real users have a large deviation in their ability to process information. Some are visual learners, some are analytical, some are contemplative, some are driving personalities,

So when you are load testing, it will be beneficial to have a randomized pause between steps in your scenario. This will realistically represent the length of time from when a person receives a response (page) from your server to the time that person requests a new page.

I used to believe that interpolation for performance metrics was acceptable. However, now I have found that I was wrong. Especially when it comes to think time.

How Do We Know If We Are Successful?

One of the over-arching objectives of load testing is to measure system capacity. When you are load testing, what are the goals? Generally, everyone wants to know how much load (user traffic) the system can handle without significant performance degradation. What is significant degradation?

I've known some web Product Managers that require at least one 72 hour load test on a monthly basis. In fact, a couple are customers of LoadStorm.

The reasoning behind conducting such long tests is that memory leaks can be very difficult to find in a 1 hour test and should be detectable after 72 hours. Comparisons of memory consumption in the first hour to the 72nd hour under the same load may expose any defects in the code that leave memory wasted.

CTOs Don't Get It

One of the most disruptive events at organizations is the disconnect between decisions made by managers and the views of those decisions by employees. Does this situation sound familiar?

I've seen too many people ignore load testing until it was too late. Some don't test at all. Some treat it with a duct tape approach: Test a little here, test a little there, no planning, no consideration for what is a real-world traffic scenario, and sloppily thrown together at the last minute before going live in production.

I've have always liked Gary Busey in movies and on TV. He has starred in everything from Kung Fu to The Buddy Holly Story to the Simpsons to Gunsmoke to Saturday Night Live to Lethal Weapon. His recent appearance on Celebrity Rehab with Dr. Drew showed just how much brain damage he suffered in that motorcycle crash. In some ways, Gary's life is similar to a good stress test: Ramped up the volume until the system could not respond appropriately to requests made of it.

On to the good stuff about system performance testing.

Performance Metrics Tied to Business Metrics

Mike Kelly at SoftwareSearchQuality.com has an article about developing a working understanding of the business drivers for application performance. He postulates that a skilled performance tester can build better tests through understanding performance goals better through knowing what the business truly needs. He also believes this understanding makes test results more actionable. The key question to ask is, "What do the project stakeholders care about?" Read more about
application lifecycle performance testing and monitoring strategies.

In an effort to identify as many performance enhancing suggestions as possible, here are a few more ideas that are specific to Microsoft's IIS web server. I will try to add to this post whenever I find new IIS performance improvement ideas.

Of course there are hundreds or thousands of posts out on the web about performance testing. I thought I would share 17 good sources focused on web application performance testing. Some of these are lists that lead to other excellent posts.

Ensuring Web Site Performance – Why, What and How to Measure Automated and Accurately

Performance Testing Articles

On this day in 1994, Israeli Prime Minister Yitzhak Rabin and PLO Chairman Yasser Arafat reached agreement in Cairo on the first stage of Palestinian self-rule. This was the first agreement with the PLO that acknowledged Israel's right to exist.

Apparently, not everyone agrees with my buddy Steve Jobs. That seems obvious to me because I have a large number of friends and colleagues that are very much in the Microsoft collective. However, I was quite surprised to find that a Netscape hero would be taking a firm public stand against Steve's State of Web Development and defend IE so vehemently.