Rodney Bone is a performance consultant that works for Revolution IT in Brisbane, Australia. He has graciously invested his time to share insights about performance testing interview with us. Please follow him on Twitter (@rod_bone) and tweet your thanks for his interview.
That’s Rod in the picture at the right when he did some reserve time during the Brisbane floods over Christmas. The slow sign is just ironic.
What is your technical background?
Started as a software engineer with Accenture, where I was involved in the entire end to end SDLC including experience in the BA space.
Do you consider yourself more of a software developer or QA professional?
Now, definitely QA, I think todays generic developer is only one part of a large picture and Accenture’s model of exposing developers to the whole SDLA is one that should be encouraged by all companies.
How do you determine the load to apply to the target app during load testing?
Ask the business, and ask as many people as applicable. Users, and BA’s have different opinions, so talk to them all. Once you nail the processes and the amount of transactions per hour run with that. Back that up with Log info as historical information is the best source of information. With new applications this is not always available.
Do you prefer using requests per second (RPS), concurrent users, or some other metric to define load?
They answer different questions. You can sometimes hit a server with the right RPS with one virtual user but that doesn’t tell you that the web server is up to serving the required amount of concurrent connections. I have tested an app where they plugged the Apache Load Balancer in out of the box and it only served 20 concurrent connections. FAIL! And you wouldn’t pick this up without concurrent users.
Do you see much difference in load testing for web applications versus traditional software?
Yes. From my experience, in traditional applications, the developers know more about how it all works, if it’s in house then developers are easy to access and know the environment. With Web and CMS, there is so much the developers don’t know about, this is especially true when they integrate out of the box solutions. So many software developers now are using solutions they buy from someone else and they are slow to turn around bugs, functional or non-functional. Obviously this depends on a lot again, traditional applications can also fall into these traps, but it’s more common with Web.
Do you feel like performance testing is an accepted critical part of the development life cycle?
It is getting that way yes, with more and more crashes getting exposure on the news here and with it happening more and more it has become a critical part of testing.
What are the key KPIs you track for performance testing and tuning?
Obviously transaction response time, memory usage, disk space, and CPU time.
Tuning: network delay and stored procedure times.
How do you use them?
Response times usually relate to SLA’s so that’s a given, memory usage is an indication of scalability and capacity, as is disk usage and CPU. I also look at Memory during soak testing to look for memory leaks.
Network delay and SP times are something you look at once you identify the application is slow to determine where the bottleneck is.
What are some of the most common web app bottlenecks that you find and/or fix?
Common: misconfigured server, poor performing stored procedure, or the application. Less down to infrastructure as most places I have been they spend a lot of money in this area.
Do you ever measure web performance in terms of revenue decrease/increase?
I read the other day “Every 100ms of latency costs Amazon 1% of profit” I always keep this in the back of my mind when consulting; the client needs to understand their risk.
http://radar.oreilly.com/2008/08/radar-theme-web-ops.html
Have you ever measured application performance re-engineering impact by operations cost reduction?
Yes. I tested a workflow application, and the speed of this application had an impact on both the productivity of a large portion of their workforce, and the performance of their helpdesk staff, which had a direct effect on their reputation.
How much involvement do you have with load and performance testing?
Its been what I have been doing for 3 years now, the last 2 using LoadRunner, before that mainly capacity and manual load testing batch processes.
What do you think is the most important aspect of load testing?
As mentioned above getting the scenario right and answering the business question. No point telling them , “hey it breaks at 1000 users, when they have 100 people working there and they were concerned only with network latency. (Simplistic example I know)
Who are the most influential leaders in the field of performance engineering?
For me obviously my seniors here at RevolutionIT, I get a lot of support from them (and that’s not just a plug). You should self-educate but clients like to see consistency of methodology also.
I love “The Art of Application Performance Testing: Help for Programmers and Quality Assurance” by Ian Molyneaux. It’s a great one for understanding performance testing and I recommend it to people who ask me about getting into performance testing. If you can get through that book and still be interested, then you’re up for it.
When and why did you get into this industry?
I was in the military and liked the idea of spending more time with my family. IT seemed like a place I could be creative and where I could achieve a better work life balance. Why testing and in particular performance? Because I saw it as a niche where I could make a mark through hard work and self education. As I got into IT in my 30’s I needed to get away from competing with Uni grads who had been “geeking it up” all through School.
What do you believe to be the most critical elements of web application testing? Why?
This has to be answered in two areas. From a business perspective you have to understand the business risk and the question they want answered in order to ensure your testing is relevant. From a technical perspective you have to ensure you are placing the right amount of load from the right place and be touching all the right back end systems. That sounds like performance testing in a nutshell and probably because it is. You leave something out and you have wasted your time.
How do you see development or testing evolving over the past few years? the next few years?
I see a rush to Agile here in Australia in an attempt to catch up on the IT demand. However in my own opinion this is a limited approach, a good team will achieve with any process and seeing V model work and agile not, I am not completely sold. I see agile staying in the web space, and more complex deeper projects going back to V model type processes. Agile does not work with high churn, and churn comes with IT these days. That’s just my opinion.
Please share with us an interesting story about your experience with software development or load testing.
Actually this is not a load testing story as such but the best one I have relating to IT. My father in law signed up to an internet plan with one of the providers here I have very little time for due to their lack of support. So that sets the scene. He was complaining to me that he got dropouts, and when he was on, very slow speeds.
The first thing I did was run a speed test from ZDnet and this verified his connection was shaky and slow. Ping tests also had various results. I plugged my laptop in (this was back in the day of inbuilt dial up modems), to make sure it wasn’t the modem. I got the same result.
So called the provider and they said everything was fine. Insert heated argument.
After almost half a day of head scratching, my sister in laws boyfriend unscrewed the telephone jack on the wall and found a dead squashed cockroach. Moral of the story real bugs still exist! And ALWAYS think outside the square.
If you could make a career from one of your favorite hobbies, what would it be?
Rock Star!