Load Testing in the Cloud

Response Over 6 Seconds is the Top of Your List?!

I read an article today on the E-commerce Times site Web Performance Metrics That Matter. It was the first result in a Google search on "web performance metrics", and the title sounded like a perfect hit. My intent was to see what other people think are the best performance metrics.

Here is a paragraph that really surprised me:

Even among top sites, performance differentials can be significant. For example, in Keynote's August 18th e-commerce rankings, the site with the fastest aggregate response time -- EddieBauer.com -- responded in an average of just over 6 seconds, while number-two site Amazon.com took about 11 seconds. Office Max occupied tenth place in the rankings, with an average response time of more than 24 seconds -- nearly four times that of the best performer.

So let me get this straight...you can be in the top 10 of best performing sites and have an average response time of 24 seconds?! You've got to be kidding me.

So I went back to the article. Then I noticed that the publish date was September 3, 2003. Six years ago...that's something like 91 years in Internet time. Still, that is the top result in a Google search. Wanna guess what the #2 search result was published? February 3, 2004 - about six months later.

My brow was definitely furrowed at this point. I couldn't get my head around the fact that web performance metrics, as a topic for discussion, haven't been more up-to-date. Could Google just be blind to the good stuff? Maybe.

I looked further down the list. Web Test Tools was the third result. It links to a very useful site with resources about load and performance testing tools, but I didn't see anything about metrics.

The next three results were about the same book: Capacity Planning for Web Performance: metrics, models, and methods. Sounds like a cool book, but I was looking for online content rather than an advertisement for the bookstore.

Number 7 on the top Google results was an article called Know These Three Performance Metrics to Increase Website Sales. You guessed it - all about sales metrics.

  • Conversion rate
  • Cost per Action
  • Value of a Buyer

Although these are important, they weren't what I was looking for. I was searching for application performance reflecting the effectiveness of the website to deliver what the user needs.

Number 8...this was starting to ring in my head like a David Letterman top ten list...was a Forrester report they are selling for $500. It appears to be more about figuring out the KPIs you need to run your business than website performance.

Number 9 was very similar. An article entitled Five Performance Metrics describe conversion rate, cost per visitor, cost per lead, cost per customer, and value per visitor. Interesting, but that's still not what I'm looking for.

Finally, number 10 (I'm stopping at 10 for now) is Understanding Web Performance Metrics. Sounds good? It's covers "statistics about the behaviour of web site visitors". Nothing new and not very informative. This article is an example of a good title without much meat on the bone (kinda like this blog post).

Web Performance Metrics Article

My conclusion from the top 10 search results? I need to write an excellent article about response times, error rates, throughput, requests per second, and anything else I can find that is relevant.

Now that I have a topic, be watching for more of my rantings.

Improve Performance

Looking for a simple way to speed up your site? We found a cool performance solution that can deliver HUGE improvement without rewriting your web application. aiCache web application acceleration is a new partner with LoadStorm because they can help our clients where we cannot. It's a great alternative to buying biggers servers, implementing load balancers, or re-architecting your app.

Placing aiCache in front of your web servers increases the number of requests per second and concurrent users you can sustain by orders of magnitude - delivers in excess of 250,000 HTTP requests per second while managing hundreds of thousands of connected clients.

Woot.com uses LoadStorm for performance testing and aiCache to handle the crush of traffic. They have saved a bunch of money and eliminated time spent optimizing.

more information

New Pricing Model

Storm on Demand Users Cost
250 $9.97
500 $19.95
1,000 $39.90
5,000 $199.50
10,000 $399.00
25,000 $997.50
50,000 $1,995.00

To See All Plans & Pricing Details

Web Developers Like Us!

“I deeply resent every second of my life I waste by thinking about load testing. All I ask is that our site be tested with a lot of traffic and without a bunch of BS for me to deal with. Thanks to LoadStorm, I need never again lose another moment to this insufferably tedious aspect of my job. I can feel the rage melting away.” - Shawn Miller, Web Developer, Woot.com

"I am definitely a fan of LoadStorm. I like its ease-of-use and the way in which the solution scales." - Darin Creason, Sr. Software Engineer, TransCore Corp

"LoadStorm is a great product at a great price with fantastic support. Setup time is minimal and the learning curve is low which was essential for us. I couldn't be happier with LoadStorm! Thanks again for your excellent support and product. - Nate Woolls, Director of Software Technology, InstallerNet, Inc.

"You guys are the best. Great customer service." - Melinda Keedy, Windstream Communications