Performance Testing Interview with Ricardo Sueiras – LoadStorm
Ricardo Sueiras is a self-professed “IT professional and geek” (found that on his blog. His formal title is J2EE Systems Architect at PricewaterhouseCoopers LLP in Canterbury, United Kingdom.
Ricardo has extensive experience with web performance and load testing, and we are grateful he invested his time for a performance testing interview to share some good thoughts with us about a topic we are excited about.
What is your technical background?
I am a system architect, working in J2EE for around 10 years, with a focus on infrastructure design and architecture.
Do you consider yourself more of a software developer or QA professional?
Neither I guess – I am the guy that has to put all the pieces together, make sure it all works and performs as required. When it doesn’t, I figure out why and what needs to be done!
When and why did you get into this industry?
I’ve been in IT for over 20 years as it has always been a passion – it is hard to think of any other industry to be in for me, and am still as interested and driven as when I started 20 years ago. I think its IT’s ability to evolve, change and for new technologies to come along that help keep it fresh and interesting.
What is your specialty? Why?
I tend to specialise in open source and collaboration/social software. I have implemented over a period of years a robust process within our organisation for the adoption of open source tools, platforms and software and am heavily involved in collaboration software, initially with the first wave (cc:Mail/Lotus Notes) and now onto the new wave of social collaboration tools. I am also interested in load testing, and have worked on load testing a number of our internal web infrastructures and applications.
What do you believe to be the most critical elements of web application testing? Why?
I can only really comment about my experience in the corporate area, and here I think I would say that for me, actually doing the right level of testing is the single most important factor as for many projects I see, testing is always seen as an optional extra and when project timescales looms, appears the first thing that gets trimmed – a wrong decision in my view. That aside, thinking about the projects where testing was really effective and added significant value i think the common elements were: (a) getting a team who knew the end to end architecture really well, (b) having the appropriate levels of access during testing (c) having good monitoring and measurement tools during the tests to get feedback on how the critical components are working (d) having solid test scripts written by people who have spoken and engaged with the business/users (for new apps) or examined logs/usage patterns (for existing apps) and finally (e) having the appropriate testing infrastructure in place.
How do you see development or testing evolving over the past few years? the next few years?
For me, the significant improvement over the past 5-10 years is that functional and load testing is becoming a better understood process and that the business understands why we have to do this. I can remember a time when web applications were just launched without any consideration to performance testing, and little functional testing other than a few test scripts.
The future for me is very exciting, because of what people are doing with cloud computing performance testing tools. Currently, load testing is a significant investment and requires dedicated resources which are not always being used. We (PriceWaterhousCoopers) use loadrunner with 100K users I believe, and its NOT CHEAP licensing, and the hardware to support this is definitely not cheap either!
I was first made aware of the potential for load testing via cloud services via Billy Newport at IBM, who blogged about using EC2 to do cheap load testing some years ago. Services such as LoadStorm to me are the logical next step here, and I think there is the opportunity to significantly decrease the cost but also increase the flexibility of testing using these kinds of emerging tools.
Is there anything commonly overlooked in web application testing?
Yeah, for me I think the single most overlooked thing is that before testing even starts, the business/application sponsor do not have a really clear idea of what they actually want in terms of performance, or do not have accurate data as to how the app may being used. This leads to a disconnect between writing the test scripts (functional and load) and then test data obtained is of real value to the organisation and leads to an incorrect perception and mismanaged expectations (i.e. the old adage, the app has passed load testing….but what was tested?)
How much involvement do you have with load or performance testing?
Lots! I am probably involved in load testing for most major projects and infrastructure we deliver.
What would you say is the difference between load testing and performance testing?
Simply put (for me) the difference between measuring the end user response and the corresponding impact on infrastructure to support various workloads.
What are the KPIs you track for performance testing?
- Throughput
- Response times
- Errors
How do you use them?
We use these stats to gauge where likely bottlenecks are and steer our investigations.
Who are the most influential leaders in the field of load/performance testing?
There are a lot of good reference blogs out there sharing great and good information, so I would like to say “You” are influential as the more people blog/write up/share about this stuff, the more load/performance testing becomes demystified.
Who are the top testing experts that you know?
Billy Newport
WebSphere and J2EE Blog
What is the biggest change you have witnessed in the way people conduct load testing?
To be honest, in my org I have seen little change. There still seems to be resistance to doing it, and also some degree of mystery as to what goes on. My personal belief is that the load testing teams need to be a lot more proactive in the enterprise and sell what they do in a way that most people can relate to. I think this would help a lot to lower the barrier to acceptance as part of the deployment process.
Do you feel like load testing is an accepted critical part of the development life cycle?
No, not really. In fact, even in our organisation load testing is still “optional” (whereas functional testing is mandatory).
In what ways have you seen test automation tools evolve in the last few years?
I would say that this area, at least in my org, has really taken off. Unit testing, in conjunction with moving to a more agile dev process, has become more accepted and understood. We have champions and gurus but it is still not implemented/done in most projects. I do see that changing however, especially as we move to automated builds which will not accept apps that have no tests!!
How does cloud computing affect the future of automated testing?
I see this having a really positive impact and hope the stuff happening at LoadStorm takes off.
What resources do you typically remove/restrict from a system to perform a stress test?
what we typically do is run a test on half the infrastructure to mimic one datacentre going down.
That wraps it up for this interview. Once again, thanks to Ricardo for sharing his experience in performance testing with us! You can follow him on Twitter as @094459. Tweet him a thanks for his thoughts.