Today I read an article by Omer Brandis on IT Toolbox about SAP stress testing versus performance testing. Maybe his perspective differs from mine because he is not focused on web applications. I just haven't seen other technologists use the definitions and goals of the two types of testing like Omer uses them.

I'm reading the classic book The Art of Software Testing by Glenford Myers. While it may be old (copyright 1979), it is still full of golden nuggets. Myers uses real examples from the ancient days of software development to demonstrate the practical side of testing.

One of my favorites is his definition of stress testing.

"Stress testing is the process of evaluating failure points in a web application by introducing adverse conditions into the runtime environment."

I thought it would be fun to see how Wikipedia defines some of the key terms we deal with in web application development and testing.

A web developer called me the other day and asked, "Is load testing the same as performance testing?"

I pondered for a moment and replied, "Not really." Hmmmm...now I needed to support my answer. Thus began a hunt for good answers to my client's question.

Searching Google, gleaning from our interviews, and asking respected colleagues were parts of my pursuit for the truth about load vs. perf. Sorry to abbreviate, but I've heard cool engineers shorten it to "perf", and I desperately want to be cool. (in a geeky way)

It's that time of year. Set some new goals. Make some changes that will have 2010 run smoother than last year and accomplish more than ever before.

Two key parts to this post:

1. Performance Testing Frequency
2. 8 Principles to Succeeding in Your New Year’s Resolution

In the spirit of New Year's resolutions, I'm going to try to provide as many tips of the week as I can. There are weeks when I might get too busy, but I'll sincerely try to post a small hint or trick that will be helpful to web developers. Tips may be about load testing, performance testing, performance tuning, stress testing, or anything related to web application development.

Performance Reference Point

As I sit here on this Monday morning and try to figure out what I should do first, it hits me that today is the beginning of an exciting year. 2009 will probably be remembered by most people as the big global economic recession. For me, 2009 will be significant because of the launch of LoadStorm.

Your website is a little slow - so what? Well, it is probably costing you money. I have been researching published facts about web performance because we are always trying to understand our industry better. This post should help you realize that improving your web application performance can directly impact your bottom line by 10% or more. Don't believe me? Read on...

I read an article today on the E-commerce Times site Web Performance Metrics That Matter. It was the first result in a Google search on "web performance metrics", and the title sounded like a perfect hit. My intent was to see what other people think are the best performance metrics.

Here is a paragraph that really surprised me:

What is the Slashdot Effect?

The Slashdot effect occurs when a smaller website is flooded with visitors, usually as a result of being linked to from a high profile website. The name comes from the Linux and Open Source blog http://slashdot.org, a site which handles 80 million page views a month. Many home servers or low bandwidth sites that post content interesting to one of these larger blogs have experienced significant downtime when the traffic from Slashdot suddenly migrates to their own servers. Performance engineering for the Slashdot effect is essentially engineering for a worst case scenario - but that worst-case involves more traffic than developers could ever have imagined!

There are some excellent stats gathered about performance testing at this resource site.

I especially like the one that states 5 second response time is the cutoff point for the business doing well. If the app takes longer than 5 sec to respond, the company employees start to get frustrated. It makes sense to me that productivity will go down. That not only leads to unhappy employees, it invariably leads to lost customers and lost profits.

Ricardo Sueiras is a self-professed "IT professional and geek" (found that on his blog. His formal title is J2EE Systems Architect at PricewaterhouseCoopers LLP in Canterbury, United Kingdom.

NetworkWorld's Senior Editor Denise Dubie provides her views on What is considered good application performance?

The article essentially concludes that whatever the end user can tolerate is probably adequate performance. I was hoping for more technical content and details of studies or something like that.

Don't Get Caught With Your Britches Down (hat tip to Capt Kirk in Wrath of Khan)

Many CIOs do not embrace cloud computing. They have pressures from stakeholders to make system service levels "five nines", and clouds don't offer those SLAs. The trade-off is between highly redundant and available systems at a high cost to maintain compared to much lower cost without the guarantee of high availability. Cloud server platforms are relatively very inexpensive, but it isn't perfect.

On this day in 1992, a lack of load testing and poor software quality assurance practices possibly caused as many as 45 deaths in London.

Ok, this comes under the heading of "weird stuff" and it is a stretch to call it load testing. My thinking is that this guy is running a test to see how many Madagascar hissing cockroaches fit into his mouth. I don't think it is stress testing because he isn't trying to exceed the capacity of his mouth. In stress testing, he would be trying to break the system; however, in this instance of the test, he is merely trying to break a record (11) previously set by a guy in Kentucky.

Queueing Theory and The Science of Waiting

The more you learn in the field of web development (and technology in general), the more that the scenarios in 'Num3ers' seem possible. 'Rubbish!' I thought when I first saw the show. 'Maths can't predict that! There are too many variables! There's human error to account for! You'd have to be omniscient to calculate that!'. Well, queueing theory is making me start to think about whether there is a literal way to eat my words. Web developers use it to do things like calculate the probability that a given amount of traffic will arrive at a website at any given time, and to estimate delays in each stage of the queue, from accepting a request, to waiting to be served, to the time taken to actually be served. We look at what queueing theory entails.

Today we're looking at some server overload circumstances from the sad, to the silly and the very expensive ... and some where the hackers got the blame for something that may well have been the load tester's responsibility!

Website flooded after heavy rains

The following is an interview with David Makogon, and he shares his thoughts with us on load testing.

Anyone know how I can find some cool things that happened in information technology on this day in history? I tried to Google it, but the results are useless. However, I did find out that today is Blog Action Day, which is an annual event held every October 15 that unites the world’s bloggers in posting about the same issue on the same day with the aim of sparking discussion around an issue of global importance.

On October 14, 1962, six thousand (yes 6,000) Unification church couples got married in Korea. Think about it...that is probably the largest STRESS test of all time! :-)

Before we get to the software testing news, here are some other events that happened on this day in history that I find interesting:

• Space probe Magellan burns up in atmosphere of Venus in 1994
  

Today is the 67th birthday of a musician who has been relevant in the mainstream for longer than virtually anyone else in modern music. Paul Simon was big when I was a child and still has a strong following. I remember the tunes of Simon and Garfunkel playing in our house before I even knew any of the entertainers. "I am just a poor boy..." was more than just a song, it was my story.

1. Test against your production environment.

I told this to an "expert" at the Association of Software Testing conference this year. He response was something like, "Are you kidding me?!! You'd have to be stupid to load test your production setup."

What occurred to me later in the conversation was that this guy works for a Fortune 100 company and has a six-figure annual budget (not counting his salary) just for performance engineering.

Performance testing is important. Web developers never want their sites or applications to crash under load. If they do, so what? Not all of the failures below are related to high traffic, but the failures are notable because of their impact on users and the far-reaching ripple effect.

From promising thousands of customers £7 top-of-the-line computers, to nearly causing an 800-plane pile-up, 'inconvenience' doesn't even begin to describe some of the most famous application failures.

I saw a tweet today that referenced web performance improvement, and that linked me to an interesting post on Developer.com.

A Little Cache Goes a Long Way

Drupal is the target of load testing for this series of articles. If this is your first time reading any of the articles in this series, please review the introduction and summary called Load Testing Drupal for a good context of what we are doing.

Drupal is one of the most popular open source content management platforms and is being adopted at ever larger organizations because of its powerful features.

Our team is conducting load testing against copies of Drupal on various sizes of EC2 instances. We intend to establish performance benchmarks for Drupal version 6.x.

Everybody has problems. As web developers, we are trained to find solutions using technology. I submit to you that the most important things (all great lists come in threes) when solving a web problem are:

1. Simpler is better
2. There is always a way
3. Passion overcomes intelligence and skill

Simpler is Better

We at LoadStorm are excited to announce that we now have 2,000 customers!

In order to celebrate this milestone, we are hosting a contest for the best LoadStorm feedback. WOULD YOU LIKE TO WIN $250?

We are eager to hear from all of our customers so that we can make LoadStorm a better fit for everyone's load testing needs. Please submit your best LoadStorm recommendations, frustrations, wish list, or any type of feedback that you believe will help our developers improve our product.

I was in a meeting this morning with our development team, and we have a desire to run some load tests against Drupal. We want to find out how Drupal performs on a standard Amazon EC2 small instance. We believe LoadStorm can do a good job of stressing an out-of-the-box Drupal application implementation. Not sure if we want to use a LAMP or MS stack yet, and we are open for suggestions.

Anybody interested in this type of tech study?