In this email interview, software testing professional James Christie, BA MSc MBCS CITP, and ISEB accredited Software Testing Practitioner, talks about his views on usability testing, load testing, test automation, and off-shoring.
My thanks to James for agreeing to share his expertise with us. James has 24 years of IT experience, and he is dedicated to providing professional testing services to his clients in order to deliver high-quality software. I follow James in Twitter and have enjoyed his insights into the testing industry.
Load tests identify performance problems that will affect users. If the web app takes 15 seconds to respond when someone is putting a widget into your e-commerce shopping cart, the probability of that user inputting a credit card number decreases dramatically. Cognitive psychologists have show distinct correlation between user confidence and unexpected application behavior. When the system does something “odd”, then people don’t trust it. Slow is bad. Slow response loses visitors and loses revenue. It can cost your company $ millions (NO EXAGGERATION)!
Mr. Prickett, my 12th grade English teacher, was one of my all-time favorites. One day while I was arguing with him about why he took off points on my paper, this popped out of my mouth: “I assumed you knew the character because it was part of the assignment!”
He proceeded to walk to the chalk board (I’m old, before whiteboards), and he wrote “A S S U M E” in huge letters. Now he had everyone’s full attention. As he explained, he circled the letters for emphasis, “for U to assume is to make an A S S out of both U and M E!
I’ve forgotten most of what he taught us, but I’ll never forget that lesson. So I decided to not assume that people coming to our sight will know much about load testing. Hence, this post. I hope you find it useful because it isn’t intended to talk down to anyone.
There is too much to say about load testing in just one blog post, so I will break this down into the key questions: what, why, who, when, how, and where. Let’s start with “what”.
Somebody sent me a link to a YouTube video that was recorded during a conversation with Verizon customer support. The issue is about calculating the billing.
Apparently, Verizon is overcharging the quoted price by 10,000%. And the worst part is that they don’t understand why that is wrong.
I was cleaning up some stacks of files on my desk last night. In a pile from my previous job, I found some notes taken during a CSIA event where a gentleman from Forrester Research was moderating. My recollection is that this event was about September last year.
Anyway, here are a couple of interesting nuggets I wrote in my notes:
Beginning of the tornado that struck northwest Oklahoma City and Edmond on Feb 10, 2009. This picture taken about 1.5 miles southwest of the first touch-down point at NW Expressway and Rockwell.
Taken in Oklahoma City, OK by peknapp on Feb 10, 2009
Recently I’ve become aware of the existence of videos on YouTube relative to the software testing industry. Ok, I realize that I’m not cutting edge. You may have been watching these videos since before Andy Griffith went into syndication. Yeah right.
Anyway, I thought it would be nice if someone had screened out those that are difficult to understand or lacking in relevant content. After scanning through several of them today, it is obvious to me that I have a much different point of view than some of the reviewers that give most of these a thumbs up. Some are good. Many are not. I hope by pointing you to a few of my favorites over the coming weeks, you will gain some value from my wasted time and spare you the pain of watching all the bad ones.
Here’s a video presentation by one of the most well-known gurus in the software testing industry – James Bach. The posted summary is below the embedded video. Please let me know if you find this helpful.
This morning, a colleague sent me an email linking to a TED video of a talk by Dr. Kary Mullis about Celebrating the Scientific Experiment. I found it interesting for several reasons and want to share some of my thoughts as it relates to software testing. The global warming comes in later. Keep reading.
Here’s the published intro to Dr. Mullis and his talk:
“Biochemist Kary Mullis talks about the basis of modern science: the experiment. Sharing tales from the 17th century and from his own backyard-rocketry days, Mullis celebrates the curiosity, inspiration and rigor of good science in all its forms.
Kary Mullis won the Nobel Prize in Chemistry for developing a way to copy a strand of DNA. (His technique, called PCR, jump-started the 1990s’ biorevolution.)”
comScore’s annual Super Bowl research, which included both pre- and post-game surveys, measured various aspects of media use and ad preference during the big game.
Denny’s had the third best brand improvement score of 39 percentage points. That should be good news, but the extra attention crashed their site. This article by Download Squad states, “Marketers need to learn to anticipate promotional effects on servers, end of story. Don’t write a check your host can’t cash, basically.”
Denny’s site wasn’t the only one overwhelmed with traffic, Vizio’s ad on the big game was specifically created to drive web traffic through a “free TV” contest. Yep, dead under the load.
An expert is a person who has made all the mistakes that can be made in a very narrow field.
– Niels Bohr
Where have all the load testing experts gone? How do I find them? Who claims to be a load testing expert anyway?
I’ve met many people that are software testing professionals. Some of those consider themselves experts. And indeed, many of them are.
At a breakfast meeting yesterday (DCC Iron Hour), a gentleman asked me what I do. “Load testing” I told him. “We have a load testing tool that runs on a cloud computing infrastructure.”
He looked puzzled. “Are you load testing airplanes?”
It took a few seconds to realize he was trying to make sense of “cloud load testing”.
Maybe this is more what he was thinking…
Load testing tools can apply large volumes of virtual users against a web application. In its simplest form, LoadStorm generates traffic that goes to a site, searches the home page HTML for links, randomly chooses one and activates that link. The tool carries out a series of these HTTP requests for as many simulated users as the tester requests.
That simple load testing scenario produces results with KPIs such as error rates and response times that the tester can analyze. However when AJAX enters the equation, load testing gets a bit trickier.
I recently found and joined The Software Testing Club, an online community of software testers. The site has some very good video interviews of leaders in the industry.
Consistent with the “storm” theme of photos on our blog, I found this one through checking out web developers to follow on Twitter.
This photo was taken from the Hubble Telescope. It looks more like a cartoon drawing of someone’s concept of heaven, but it is real.
According to a blog post on Web Service Guard, downtime costs money and is considered by execs to be a “complete disaster”. That may seem obvious to most of us developers/testers, but what is cool about the post is the summary of statistics.
A headline in one of the LinkedIn testing groups caught my attention this morning. Sergey Lesnikov posted a new discussion thread titled, Top 25 Programming Errors for Software Testing. It is on the Software Testing – ISTQB/ISEB group.
Quick summary:
- International collaboration from cyber security organizations
- List of bugs that enable cyber crime
An article caught my attention because I follow the author on Twitter. Two days ago, Jeffery Way published 15+ Tips to Speed Up Your Website, and Optimize Your Code!
Questcon Technologies hosted an excellent webinar this morning about how testing in Agile is different than traditional development. Officially titled “Assessing Agile Test Teams: Adoption & Effectiveness, the material was presented by Shaun Bradshaw, Director of Quality Solutions. Shaun is very knowledgeable and has Agile certifications (sorry I didn’t write them down).
My top 10 tidbits from Shaun’s discussion:
We posted an article on the LoadStorm blog last week about Software Testing is Detective Work. We covered about 50 questions that could be very useful to ask oneself when engaging in a testing process. Those questions were rather generic to the world of software testing, some were just intended to be funny, but they weren’t specific to load testing.
If you want to lower the cost of your software products and speed up your development time, then invest more resources in quality assurance from the beginning of your project.
When I first encountered that concept about a decade ago, it caught me off guard. I had a similar reaction to when a backpacking book stated, “If your feet are cold, put on a hat.”