“I find it rather easy to portray a businessman. Being bland, rather cruel and incompetent comes naturally to me.” – John Cleese
Rick Martin of Bearblu told me the other day that he had wrapped up a testing gig for one of top 3 Indian tech companies. Actually, his client was a U.S. Fortune 500 company that used the offshoring for development and testing. Rick is a highly skilled senior manager for QA, so he was providing the leadership for software testing on the project.
It depends. Stress testing tools can be invaluable for finding the breaking point for a web application. But even the best tool and test plan won’t fix the problem.
The purpose of stress and load testing is to discover. Measuring response time and errors relative to user volume can point to solutions; however, the developer(s) must engage in performance tuning to eliminate bottlenecks or otherwise improve throughput.
The wealth of information on scaling is profound. Although, there was one lesson that I found to be the most interesting. This is that every web application is different. This is not to say that we cannot learn from others that have come before us, but we need to understand that every application has a wide variety of demanding requirements. This may include a special read/write mixture of database operations or an inability to effectively cache content. Because of these differences, developers must be extremely perceptive of their system. This means constant monitoring of the site’s trends.
What are the repercussions of a Software as a Service outage? Obviously, there is significant monetary value being lost during these periods. Customers may not make purchases on eCommerce sites or could simply take their business to a competitor. The basic sources of downtime are attack, overload, no power, bugs, and breakdowns.
Most people believe that if it ain’t broke, don’t fix it. Software engineers believe that if it ain’t broke, it doesn’t have enough features yet.
How a corporate entity reacts to crisis is important. Users expect official representatives to have frequent and forthright responses to crisis. Additionally, it is important that official representatives of the company do not make overly revealing or accusing remarks. It is rational for users to want 100% success rate in a company’s product or service. However, it may not be possible to deliver that promise. The reaction to service outages resulting from scaling must be handled in the same fashion.
Last year we saw plenty of downtime from Twitter. The response from Twitter was ‘growing pains.’ Since then, we have seen a large supply of posts from the development staff at Twitter describing issues and concepts related to their up-scaling efforts. Many users posted on the need for a push messaging system. Other users blame the foundation of Twitter’s problems squarely on Ruby on Rails’ supposed failure to scale. Twitter does not appear to want to change or remove Ruby on Rails any time soon. It is not my place to make a distinction on this topic.
Lightning strikes at a place in Gironde , Aquitaine, France.
The year 2008 has come to a close. Its time for blogs to tally the successes and failures of web sites and their services. It is another year of product launches and shaky web service consumer confidence. The list of failed launches is always quite high. Inadequate testing and poor anticipation of demand lead to the always popular server flood on opening day.
Thick and dark clouds form over Bristol, England. Looks like its going to rain; time to take cover. Taken with a Canon PowerShot A520 in March, 2007.
Install and configure a Magento test server using sample data from the Amazon Associates Web Service quickly and easily. There are a number of common pitfalls that can be easily avoided and some that cannot. The process of converting information from Amazon to Magento is described briefly.
Heavy rains lash Almaty, a city in Kazakhstan, a former Soviet Union Country in October, 2008. People in the picture trying to wade through the water.
As many people know, Sony’s PlayStation 3 Home Beta launched version 1.03 as an open beta to the entire PlayStation network on December 11, 2008. Yesterday, it had 6 hours of down time for the update from v1.03 to v1.04. This was in reaction to issues with an on and off ability to log in, occasional game log outs, extreme lag in rooms with many voice enabled players, and an abuse reporting system which kicked you from the network.
I was recently reading an article on Yahoo entitled, “MS: No repeat of Xbox Live holiday meltdown (hopefully)”. What immediately struck me as interesting was the fact that Microsoft is still relying on hope to avoid another black eye with their customers.
We are planning a load test of a basic LAMP Drupal target system using LoadStorm. This provides a description of the necessary environment on the system, a walk-through of testing steps, and expected results from the load test. LoadStorm does not require the user to setup large amounts of servers for an isolated test.
The Test Environment
Project studies by IBM, GTE, TRW, and several other large corporations have shown that an error in requirements detected in later phases of the software development life cycle increases costs by:
- 500% for architectural design
- 1,000% for coding
- 2,000% for unit testing
- 5,000% for acceptance testing
- 10,000% for maintenance
With such an impact on project costs, there is no wonder that few software development efforts come in under budget and on time.
From Jonathan Feinberg of IBM, Wordle is a fun tool for visualizing word distributions in articles, web pages or other media.
Rational Robot, produced and maintained by IBM, is a very comprehensive solution for load testing on web services and software. When every aspect of a sophisticated web application must be thoroughly tested, Rational Robot is an excellent choice. In addition to load testing, Rational Robot performs functional verification and error detection. Extensive scripting functionality allows the tester to thoroughly test an application. This testing can be done in a direct fashion, without having to operate through the UI of the application.
Thank you for all of the wonderful feedback during the past two weeks. Last week we decided it would be best to change the product name from aWebStorm to LoadStorm™. Several of you preferred the name, and it has some advantages.
First, LoadStorm™ sounds better. It more clearly describes the functionality of what we do. Storming your web application with a large load. So far, everyone likes it better.