Lowest Cost Cloud Load Testing Tool

Mind Games in Performance Engineering

This may not work for non-English speakers, but Cambridge University presents the following paragraph:


Olny srmat poelpe can raed this.

I cdnuolt blveiee that I cluod aulaclty uesdnatnrd what I was rdanieg. The phaonmneal pweor of the hmuan mnid, aoccdrnig to a rscheearch at Cmabrigde Uinervtisy, it deosn't mttaer in what oredr the ltteers in a word are, the olny iprmoatnt tihng is that the first and last ltteer be in the rghit pclae. The rset can be a taotl mses and you can still raed it wouthit a porbelm. This is bcuseae the huamn mnid deos not raed ervey lteter by istlef, but the word as a wlohe. Amzanig huh? Yaeh and I awlyas tghuhot slpeling was ipmorantt! If you can raed this psas it on !!

performance testing mind tricksCould you read it? Impressive! It would at first appear to be unintelligible, but it surprisingly makes sense. While this is an interesting mind trick, what does it teach us about performance testing? Well, it has been my observation that most owners of websites can understand performance metrics intuitively. Their brains can overcome the potential obstacles from their not being experts in our industry. They can figure out the scrambled world of performance engineering without knowing the code (pun intended).

Although the details about every aspect of the web application under test may not be available, with the key measurements for what a user experiences, a savvy business person can fill in the blanks rather easily. This mental connection does not require specialized training or weeks of working with a performance engineering team. They may not be able to translate Throughput in Kilobytes per Second into a meaningful correlation to the 100 megabits per second network device, yet they can tell where the "pipe is clogged" from a graph of bandwidth with very little problem.


Managers Are NOT Into Details

Professional software testers have my respect. It is difficult to be a good tester, and they usually are not recognized for the value they bring to a software development team - under-appreciated, under-paid, and sometimes overlooked. That said, many times professional testers, especially high-end performance engineers, have a tendency to describe the performance test results to executives/managers/site owners with too much complexity. The engineers want to explain every arcane detail of the process. They want the business person to see all the granularity possible, and I don't believe that is necessary.

Business people can fill in the gaps. They don't want to become experts on performance testing. Most executives just want to know that the site will withstand the load expected by the marketing department. Online Product Managers may be more interested in the performance metrics and want to see reports showing the failure points measured in Requests per Second (RPS). A site owner probably only wants to know if the number of Concurrent Users is some multiple (e.g. 10x) the normal peak volume in a day. Simplicity is key in communicating with those stakeholders that aren't directly involved on the load testing team.


Summary Version Please

My recommendation is to keep your measurements easy to understand. Provide graphs with only the most important metrics shown. Don't try to explain the test scenarios. Present very few reports. If you are going to show them request/response measurements, you should limit the rows of information. For instance, only provide the top 5 or 10 poorly performing pages or resources. It won't get you a bonus if you try to become a teacher and waste an executive's time instructing them in how the script simulates every possible user type on your site. They will just think you are boasting, or worse, they will think you are trying to bamboozle them with all your technical jargon. Either way, they won't like it.

Furthermore, when the topic turns to remediation or application tuning, be careful to start at the 30,000 foot level. Summarize the data. Identify the onerous bottlenecks, but don't try to get them to understand how the settings in the JVM or IIS/Apache or mySQL correlate to sudden performance failure. They don't really care that we are running out of database connections - they really only care that you know what to do about it.


Conclusion

The human brain is an amazing entity. It is a fascinatingly powerful creation. It can perceive activity unconsciously that most sensors could never detect. It can interpret paragraphs of words that are all jumbled and flawed. The brains of successful site owners, business executives, product managers, project managers, CTOs, and marketing department leaders are capable of seeing the big picture of performance engineering. They can see what is germane to getting the web application performing well for the required load.

Give them only what is needed and keep it brief. Communicate to them succinctly and with a focus on the primary metrics. Provide summary reports rather than spreadsheets full of raw data. Don't try to educate them on how brilliantly you crafted the load tests and found the hidden configuration setting that instantly increased RPS by 10,000%. Yeah, it's a great feeling to do your job well, however only us geeks really care about the way SQL Server slow query logs work.

Executives don't respond well to it. Let their minds fill in the gaps.




Storm on Demand - Pay Per Test

Storm on Demand Users Cost
1,000 $39.90
5,000 $199.50
10,000 $399.00
25,000 $997.50
50,000 $1,995.00
100,000 $3,990.00
200,000 $7,980.00

performance testing sign upIt's easy. You can be load testing in 15 minutes.

  1. Click the "Free Account" button.
  2. Enter your name & email address.
  3. Click the confirmation link in an email.
  4. Create a test scenario for your site.
  5. Run a load test.
  6. Analyze the test results.
  7. Send us a testimonial because you are amazed!

Customers love our load testing tool

“We needed an easy & cost effective way to load test our Windows Azure solution. Thanks to LoadStorm - highly recommended!” - Jonas Stawski, Microsoft MVP

"LoadStorm is a very useful tool." Alan Cheung, Manager - Technical Services, Dow Jones Publishing Company

"It has been a pleasure to work with LoadStorm." - Mike Compton, V.P. of I.T., Hearst Business Media

"Load-testing in the cloud was a great solution and LoadStorm a dream partner. " - Julie Hansen, COO, Publisher, The Business Insider

"There was no risk because I knew what the tool would provide before spending a dime. LoadStorm is a great tool." - Richard Ertman, QA/Release Manager, PETA

"I am definitely a fan of LoadStorm. I like its ease-of-use and the way in which the solution scales." - Darin Creason, Sr. Software Engineer, TransCore Corp

Want a Live Demo? Have Questions?

Please feel free to contact us:

(970) 389-1899

We are eager to help you with LoadStorm in any way that you need.