Lowest Cost Cloud Load Testing Tool

Comparing LoadStorm and LoadRunner

most expensive optionThanks for the replies to my comparison of LoadStorm and LoadRunner pricing. You make valid points. However, I do not see us as arguing because you are essentially agreeing with me. I have never made any negative statement about the quality of LoadRunner. David's point in the initial tweet was that LoadRunner is powerful, but for most developers the cost and learning curve make the LR solution a non-starter.

LoadStorm vs. LoadRunner is not apples to apples when comparing features. But for a product manager to make a statement that implies his tool is the only way you are going to get accuracy is simply naive at best. Blindly arrogant at worst.

Let's consider an analogy. I can buy a new Rolls-Royce Phantom for about $450,000. It is undisputable among most reasonable people that this car would be a premium solution to my transportation needs. It would provide many outstanding features and exquisite comfort.

There are alternatives. A Mercedes or a Honda sedan may be a better solution to one's transportation requirements. For my standard of living, my Honda provides the comfort, reliability, safety, acceleration, and features that I need at a price that fits my budget.

We could compare the Phantom to a Learjet (yes, that's how it is spelled on Wikipedia). My research shows that you can buy one for approximately $20,000,000. It has special capabilities that the Rolls-Royce cannot provide in a transportation solution. It's features and comfort are, shall we say, above average.

moderate optionNow, if I begin to compare these options for transportation, would I be a fool not to choose the Learjet? Would anyone argue that the Learjet is the only way I will get speed, accuracy, reliability, and confidence?

As I make comparisons, I certainly may evaluate the instrumentation supremacy in the Learjet over the other options. It possibly would be worthwhile to describe how the attention to details during the manufacturing process employed in Learjet production will ensure I have the best transportation solution. One could make the point that the redundancy and fault-tolerance in the Learjet will provide the only safe and reliable solution.

My experience tells me that such comparisons are an academic exercise that lead to nothing fruitful. In fact, you may already be laughing at the absurdity of comparing a Honda to a Learjet. Even a Rolls isn't comparable.

affordable optionWhy, then, when comparing LoadStorm to LoadRunner isn't price a fair factor in the equation? It is a fair factor in REALITY. Those of us that live in a world without a $1,000,000 budget for performance testing tools do indeed consider LoadStorm a viable option. Granted, it has limitations when compared to LoadRunner because it is designed to test web applications, not Citrix applications. Neither can a Rolls-Royce fly me to Europe. But you won't hear a sane person deriding an Mercedes-Benz E-class sedan based on its inability to reach speeds of 500 mph. Nor will a reasonable buyer pass up a Civic because it does not have retractable landing gear.

I welcome the replies and comments from performance engineers employed by Fortune 500 companies and expensive software performance consultants with LoadRunner training. You are free to point out all of the double-stitched leather, maximum speed, cruising altitude, and immaculate digitally controlled thrust reversers.

But for the love of Pete, will you please at least consider the reality of cost in the equation? Will you at least acknowledge that a sledgehammer is not always the best tool for the job?

My family is served very well by our 2 Hondas, 1 Toyota, and 1 Jeep. They all have strengths. The van is great for long vacations. The Jeep is outstanding climbing the mountain roads around here (Georgia Pass). The Civic gets about 40 mpg, and my daughter loves it.

I sure do wish I could own a Learjet. But it just isn't what I really need for my requirements.

Comments

Everyone who purchases

James PulleyEveryone who purchases LoadRunner does consider the cost, however the cost estimates that you have provided are incorrect. One can purchase LoadRunner in many models, including terms as low as a single day (Virtual User Days or VUDS) for users on existing controllers or as low as a single month for term licenses, during which a user could run as many tests as they can fit on the clock.

Cost for any performance testing solution should be measured against the risk of the application not scaling in production first, then against the cost of all capable tools, including the level of effort associated with the production, execution and analysis of tests. For some apps even the most expensive of tools (which LoadRunner is not the most expensive testing tool set out there) is a relative bargain when measured against the risk of failure in deployment.

The requirements to be successful with LoadRunner are not unlike the requirements to be successful with any other perforamnce testing tool. There are prerequisite skills in the areas of systems architecture, testing, project management and development which need to be brought to the table. As with any profession from stone masonry to plumbing to law to engineering, a period of training plus internship is highly recommended before allowing a solo effort on any performance testing effort with any performance testing tool, whether it is hosted, open source, commercial, inexpensive or rivaling GDP of small third world nations. Poor foundations + ineffective training and development = recipe for ineffective performance test efforts, poor views of your tool selection and poor views of the profession from stakeholders. This very condition is damaging the industry in pretty severe ways right now.

However, on the script development front, any developer who has strong C skills can pick up LoadRunner and be very successful, assuming a mentor available after training, within a week or less. This assumes that script development is done using a C based protocol development model versus one of the otherr languages of LoadRunner, Java, VB or JavaScript. The same short ramp holds true for developers of these other languages moving to LoadRunner if mobving into a protocol which uses this language. Development skills are the single largest barrier to successful business process reproduction with any of the commercial tools. The vendors sell these solutions as sexy record and replay solutions that any business analyst is capable of. The reality is that development skills are mandated for any in house tool for the accurate reproduction of business processes.

James Pulley

Storm on Demand - Pay Per Test

Storm on Demand Users Cost
1,000 $39.90
5,000 $199.50
10,000 $399.00
25,000 $997.50
50,000 $1,995.00
100,000 $3,990.00
200,000 $7,980.00

performance testing sign upIt's easy. You can be load testing in 15 minutes.

  1. Click the "Free Account" button.
  2. Enter your name & email address.
  3. Click the confirmation link in an email.
  4. Create a test scenario for your site.
  5. Run a load test.
  6. Analyze the test results.
  7. Send us a testimonial because you are amazed!

Customers love our load testing tool

“We needed an easy & cost effective way to load test our Windows Azure solution. Thanks to LoadStorm - highly recommended!” - Jonas Stawski, Microsoft MVP

"LoadStorm is a very useful tool." Alan Cheung, Manager - Technical Services, Dow Jones Publishing Company

"It has been a pleasure to work with LoadStorm." - Mike Compton, V.P. of I.T., Hearst Business Media

"Load-testing in the cloud was a great solution and LoadStorm a dream partner. " - Julie Hansen, COO, Publisher, The Business Insider

"There was no risk because I knew what the tool would provide before spending a dime. LoadStorm is a great tool." - Richard Ertman, QA/Release Manager, PETA

"I am definitely a fan of LoadStorm. I like its ease-of-use and the way in which the solution scales." - Darin Creason, Sr. Software Engineer, TransCore Corp

Want a Live Demo? Have Questions?

Please feel free to contact us:

(970) 389-1899

We are eager to help you with LoadStorm in any way that you need.