Performance Testing Tool Prediction – LoadStorm
I’ll go out on a limb and share what I see in the tea leaves. My research department is slightly smaller and has less funding than Gartner, but I feel very confident sharing this with the world of web developers, testing professionals, and software architects:
Legacy load and performance testing tools will lose market share quickly to newer cloud-based technologies.
Why do I feel so confident?
Cloud Growth is 400% Faster Than Traditional Software
First, I’m not the only one. Most people in our industry have been predicting this shift. It is already underway. I guess that makes me more like Gartner because they usually predict what is obvious too.
Cloud software and services are growing annually at a 27% pace as compared to a 7% growth for all IT spend. So traditional software and IT services are definitely not where the money is flowing. That’s four times faster growth. Some estimates for all cloud-based revenues by 2015 are as much as $250 billion by 2015.
According to IDC, “By 2020, a significant portion of the Digital Universe will be centrally hosted, managed, or stored in public or private repositories that today we call ‘cloud services.’ And even if a byte in the Digital Universe does not ‘live in the cloud’ permanently, it will, all likelihood, pass through the cloud at some point in its life.”
Two days ago InformationWeek stated, “It’s a fundamental shift in how we consume IT resources, much like the shift to distributed computing, client/server, and Web-based systems.”
Cloud = Scalable (and that’s very important for load testing)
Second, cloud load testing tools are much more scalable. Sure you can get LoadRunner to run complex load tests for 5,000 concurrent users. However, 20 years ago when that tool was designed, it was inconceivable to have an application with 100,000 concurrent users. Today’s web application world can spawn an unknown product that gets a nice write-up by a tech blogger, promoted on Digg/Delilcious/Reddit/Slashdot and VOILA! Huge numbers of traffic are not only possible, but expected. If your load and performance testing tool cannot easily scale to 100,000 concurrent users, then you probably have low expectations for your application or you are trapped with a tool that won’t support your needs tomorrow.
Market Forces Always Win
Most importantly, basic micro economics prove that a lower-priced product that solves a legitimate problem will replace older, less-effective products in the marketplace. Cloud solutions are much more efficient at generating load and measuring performance metrics in web application environments. Web apps are the vast majority of new software development now.
Legacy tools such as LoadRunner, Silk, Rational, and even VSTS are burdened by a client/server paradigm from yesteryear that just cannot be overcome by simply hosting it on a big server farm somewhere. Those tools were written in the 1980s and 1990s without the benefit of the new advanced infrastructure. They have an architecture that makes them inherently more expensive because they cannot utilize resources that reduce the total cost of ownership. That’s why they still are pricing their tools with legacy models. I’ve had several customers tell me that they were switching from LoadRunner because of the cost.
Microsoft probably has the best chance to keep their market share since their developer base is extremely loyal. They also tend to only use tools that are built in Redmond, and they keep their stack homogeneous. That said, we have many customers that have applications written in .NET.
Well, I don’t have a crystal ball to see the future. It seems pretty obvious to me from the trends, the buzz, and the customers that cloud-based load and performance testing tools are a hit. The numbers of people switching are increasing monthly because everybody is looking for a way to get more functionality with less cost.
In the early days of my career, large-scale software was produced only at large companies like EDS, CA, and IBM. Things change. Innovation is coming from smaller web development shops now, and they don’t typically like paying a hefty license fee to a billion dollar company (plus the 20% annual software maintenance). That model is dying.
Here’s another prediction:
The cloud model as we know it today will also die in 20 years. Maybe sooner because technology improvements are occurring in ever-shorter cycles.