Jason Buksh is a Technical Project Manager and Performance Consultant in London, England. Jason has extensive experience with performance testing at many companies including HSBC and Siemens. He is skilled with tools such as Rational, Grinder, and Performance Studio. His certifications include LoadRunner.
We appreciate his time to share some good thoughts with us about a topic that gets us excited. Here is his interview with us.
What is your technical background?
I learned to program when I was 13 – it was a vic20, I then swiftly moved onto 6502 for the BBC micro. University studying computer science was an obvious and easy progression for me. First job was writing rendering engines (C++) for virtual reality simulators. I would describe myself as a techie at heart – I’m genuinely interested in how things work. I think having a strong and long background in IT enables me to grasp new concepts easily – which is great when I have to go into different companies and need to understand their systems quickly. I’ve a 2:1 in Computer Science, ISEB Practitioner and SCRUM Master Certified.
Do you consider yourself more of a software developer or QA professional?
We should get out of the habit of separating them so readily. I feel strongly that every software developer should QA their work. Its not good enough to code and then relinquish QA to another team. Its lazy, increases delivery time, wastes effort and increases cost. Everyone should be a QA professional within their own field. I think a large dedicated QA team is a good measure to the inefficiency of an IT project. I’m going to write a post on this very topic.
How much involvement do you have with load and performance testing?
My career is built on it. I’ve performance tested many mission critical and highly transactional systems. Companies like Expedia have extremely large volumes of traffic, and the performance of such a system is paramount. My experience at global financial institutions has taught me a great deal about trading platforms and the importance of milliseconds in response time.
What is the biggest change you have witnessed in the way people conduct load testing?
There is a quiet move away from Loadrunner and its going to become an avalanche. It’s been underdeveloped and overpriced for a long time.
What would you say is the difference between load testing and performance testing?
Load testing is testing with anticipated load.
Performance testing is a generic umbrella term for several types of testing:
- Soak testing
- Load testing
- Volume testing
- Spike testing
- Benchmark testing
- Resilience testing
- Failover testing
- Stress testing
I’ve written an article on my blog about performance testing definitions that clarifies some of the differences.
What do you think is the most important aspect of load testing?
Understanding the requirements and getting the workload model correct. Your performance test is only as accurate as the model you are simulating.
Who are the most influential leaders in the field of performance engineering?
At a deep level I believe the Facebook engineers are, they get it. They scale their software better than anyone else. PHP wasn’t working quickly enough, so they wrote HipHop to increase speed, reduce datacenter space and cost. They spearheaded pagelets (Bigpipe). They measure, inspect and then improve the software if possible (True Continuous Improvement). It seems obvious, but I’ve seen a lot of companies not do this because they don’t understand their own technology.
How do you determine the load to apply to the target app during load testing?
Talk to all parties, not just one and look to evidence the requirements.
Do you prefer using requests per second (RPS), concurrent users, or some other metric to define load?
Whatever make sense in the customers context. I always think of defining the load in a way the customers will understand. You get better feedback and more accurate requirements as a result.
Do you feel like performance testing is an accepted critical part of the development life cycle?
Unfortunately in the majority of cases it only becomes part of the cycle after something horrible has happened. Then it becomes a critical part. Thats human nature in general.
What are the key KPIs you track for performance testing and tuning?
Transaction response times, Memory, CPU, and transaction failure graphs are the ones I cannot live without. Graphs can tell you a whole story simple metrics cannot.
Don’t try and replicate in software what the DB can do. I’ve seen a lot of companies try and re-invent the wheel and take out logic from the DB (such as caching) and put in their own solutions. Then they find they get into all sorts of trouble when the item in the cache is updated from an external source. They essentially start replicating the logic of the DB. Always look to tune and improve the DB – there are 1000’s of man hours invested in that technology. Your problem will not be unique and will have been solved.
Part of the problem I’m now seeing is that software engineers are sometimes just focused on the narrow field of software engineering and don’t appreciate the holistic system. Instead of getting in a DB tuning expert they may (for example) just take out items and put them in a cache – this increases the code complexity, man hours and cost.
How does cloud computing affect the future of automated testing?
It reduces the cost of ownership and makes it easier for teams to be anywhere in the world. Its a slow revolution, and its for the better.
How important is web performance to your clients?
Critical – more and more people are doing business over the web. The competition is high and people can switch easily. Customers expectations are also extremely high. Waiting longer than 3 seconds is now deemed unacceptably slow when navigating through sites.
Please share with us an interesting story about your experience with software development or load testing.
I’ve seen some pretty silly things. For example, I was called into a client that had a live performance issue that affected their whole business. They brought a consultancy in to estimate where they thought the capacity was, and then they decided to re-engineer the whole architecture at a huge cost. All without proving the consultancy’s estimate with actual testing. It was nuts. By the time they had got new hardware installed to performance test their existing architecture they discovered it could take them through the next two years of growth – easily. And this was without any performance tuning. What a waste! The sad thing is I’ve seen far more mad things than this.
Please provide a link to your favorite blog post that you have written and tell us what you like about that one.
At the moment this one about choosing the correct load testing tool. I’ve been thinking hard about how I could easily split all the load testing tools out there and explain to someone coming at it from a fresh perspective.
What are your favorite testing or development blogs (other than your own)?
I’ve come across http://bettertesting.co.uk/. Its very refreshing, informative and I like the fact that Darren is well rounded.
If you could make a career from one of your favorite hobbies, what would it be?
Personal training instructor – I enjoy teaching people how lose weight, find out what motivates them and work out how to change their body through exercise they will enjoy.