Load Testing Interview with Head Geek Andrew Lombardi – LoadStorm

Andrew Lombardi is a Java Software Engineer and Head Geek of Mystic Coders, LLC in Santa Ana, California. His blog provides insight into many aspects of technology.

Our thanks to Andrew for taking the time to share some information with us.

What is one tip or best practice that is important to you regarding load testing or performance engineering?

Keep it simple. The majority of performance issues tend to creep up when we’ve overcomplicated the codebase to the point where errors have no choice but to present themselves and hide in our tangled mess. I find that when writing code as long as you ensure each method / class covers only one concept or functional aspect of code, the testing and performance portion seem to be less of a problem and easier to “fix” if needed.

What is your technical background?

I grew up with computers and hacked on Apple’s and then IBM’s and now back to Apple’s over the years. I’ve got a Unix bent which is why OS X since Panther has really struck a nerve for me, the machine’s are beautiful and everything “just works”.

I’ve been developing with the Java platform for over 10 years now, and have also had professional experience with PHP, Python, Ruby, and of course C and Assembly Language during college.

I prefer Java because of the wealth of libraries and well designed API’s (calendar and date not included). Java is definitely verbose, but using an IDE like Jetbrain’s IDEA generally negates the amount of typing necessary to get the job done.

Do you consider yourself more of a software developer or QA professional?

Definitely more of a software developer. Just as you wouldn’t want me designing the visual aspects of your website, I take counsel with QA professionals whenever I need to ensure all around testing coverage.

When and why did you get into this industry?

I’ve been programming since I was very young, and fell into my first consulting gig right out of college around 97/98. Working with computers has always been a great passion of mine, and things evolved from there.

What is your specialty? Why?

We have usually specialized closer to the front-end tier, utilizing frameworks such as Apache Wicket and Spring with whatever persistence and business logic backend developed.

Developing closer to the front of the software stack allows you to get more immediate feedback via the visuals, and it’s also quite interesting dealing with users and the aspects involved in getting everyone to a happy state.

What do you believe to be the most critical elements of web application testing? Why?

Effective requirements from the business unit. If this wasn’t done, it really doesn’t matter what you’re testing, it’s wrong. Saying that, effective unit and integration tests, performance and load testing I believe to be most critical to the success of any new project.

How do you see development or testing evolving over the past few years? the next few years?

It seems that the past few years has seen the rise and dumb struck love of dynamically typed languages, Ruby, Python, etc, and this continues today. In the end what you get is a much more rapid development cycle, but a larger possibility of errors creeping up in your code.

Lately I’ve seen more and more companies distancing themselves from server-side web frameworks in favor of a client heavy javascript site utilizing AJAX to call back to SOA backends. The advent of 10-fold increases in javascript’s speed lately has the industry again dumbstruck.

I fear though, that putting weight on the client will not only slow down the websites, but inadvertent exposure to sensitive data is also a possibility. Call me old fashion, I think the server-side web frameworks have several more years ahead of them.

How much involvement do you have with load or performance testing?

In our client engagements we generally like to hand off to QA engineers for load, performance, and stress testing. But if this isn’t available with the client we’ll perform some of our own testing using our developer-known tools.

What would you say is the difference between load testing and performance testing?

Load testing is about throwing a volume of workload conditions to the system that we expect, and ensuring things don’t go breaking on us.

Performance testing is more about the raw data: response times, throughput, timeouts, etc.

Do you prefer using requests per second (RPS), concurrent users, or some other metric to define load?

Generally I like to measure based on concurrent users as the value gained from this allows us to estimate out how many machines are needed to serve the users we’re projecting.

Since most of our work is with stateful applications, it doesn’t matter much if we have a high request per second count, if we can only serve ten users per machine.

How does cloud computing affect the future of automated testing?

I think cloud computing opens up new worlds of possibility for automated testing. No longer is it expensive to ask that an entire production environment be cloned so that extensive load testing can be done. It’s at most a few mouse clicks and the cost is so amazingly cheap.

In the middle of last year was the best example of this. We had a major redesign of an existing site that is hosted in the cloud. Rather than having to provision new machines and duplicate all manner of IT work, we simply cloned our deployment for a staging environment so our tester and clients could attack the new design. It was an impressive use of cloud-based tech.

Any thoughts on how the global economic downturn will specifically affect the application development and testing business? (e.g. less testing, more offshoring, etc.)

There are always going to be companies that put figures on a page in a higher value bracket than the actual value brought by a local team.

The management nightmares involved in having an offshore team for development and/or testing seems to equal the cost of a local team in most cases we’ve seen. The perceived benefits just aren’t true. The pricing in certain offshore markets is making the once seemingly sweet deal, seem a little less of a benefit. It definitely can be done, and we’ve worked with offshore teams quite a bit, but you must ensure the management of those teams is done by a seasoned pro.

How much have budgets for web performance engineering changed in the past 2 years?

Unfortunately I’ve tended to see a drop in budgets for web performance testing in several companies we have consulted for. We’re big proponents of performance testing though, so a lot of client education is involved to bring the knowledge of how it will benefit the bottom line.

Do you ever measure web performance in terms of revenue decrease/increase?

We never have, but that would be an interesting metric to help prove the case that web performance is important to the bottom line.

How important is web performance to your clients?

If it isn’t important when we first meet, after our education sessions it becomes important. We make sure to explain it to them during the project’s lifecycle.

Do you see any intersection points between usability and performance testing?

In general, isn’t simple usually better for the usability of a site? An application that is built with complexity and lacks any real usability testing is most often a nightmare for performance testing.

Without clear separation of concerns, distinct units can’t be easily tested. Generally you could make the site super fast, but it will be perceived to be slow because the usability isn’t there. They are very close bedfellows.

Please provide a link to your favorite blog post that you have written and tell us what you like about that one.

I really enjoyed the collaborative writing done for the “5 Days of Wicket” series we did back in March of 2009. Several of my core developers each took a day, and I book-ended with thoughts on each. I find the process of working and communicating ideas with others to be one of the joys of software development, and this completely embodied that for me.

What are your favorite testing or development blogs (other than your own)?

• StuQ.nl – very well done and thought out posts, big focus on learning
• Enigmastation.com – well written
• letsgetdugg.com – this guy takes common components, tests the hell out of them, makes updates and increases the usability from a performance perspective.

I read a ton more but these are some of the highlights.

Do you recommend any particular white papers or webinars?

I generally prefer labs and sample projects over any white papers or webinars. I prefer getting my hands dirty, smashing some code against the debugger and watching it fail or pass based on how far my understanding has gotten.

I have found that the refcardz put out by DZone are quite good. There’s one I wrote for Wicket 1.3 available on their site.

If you could make a career from one of your favorite hobbies, what would it be?

Photography. I think I could be rather good at it as well. I’ve only recently just started taking shots with a Canon Rebel T1i DSLR, and it’s been an amazing distraction from the computer (get’s me in the daylight more). Click here for some of my photography.

What is your favorite web application or testing book that you own? That you wish you owned?

I found that for many years I would read, and re-read, Effective Java by Joshua Bloch. I don’t know where my copy is anymore, but it was always a constant in my bookshelf. I tend not to read too many technical books any longer, and instead inundate myself with blogs, articles, and ebooks, the Kindle has forever changed the way I consume literature.

That wraps it up for this interview. Once again, thanks to Andrew for sharing his experience in software engineering with us. All of the photos included on this page are from Andrew’s Flicker pages.

You can follow him on Twitter as @mysticcoders. Please send him a tweet that you appreciate his thoughts.