Software Testing is Detective Work » LoadStorm LoadStorm

“I find it rather easy to portray a businessman. Being bland, rather cruel and incompetent comes naturally to me.” – John Cleese

For me, I find it easy to portray a software developer who can also be a good system tester. My computer science degree usually gets me accepted in developer circles, although the applicability of my courses (PDP-11 Assembler, Data Structures, VMS Operating Systems, etc.) seem a bit questionable in today’s world. Some days I feel like a Hummel Collector’s Plate…old and unusual, but not quite a valuable antique.

As for the good tester part, I believe I can be as bland, cruel, and incompetent as anyone. Surely I jest. Not about being a good tester, but I certainly know that those aren’t characteristics of a good tester anymore than Cleese’s description is genuine.

Some of the best testers I’ve worked with have been very curious people. Their nature is to act like a 4 year old sitting in the back seat of your car. “Why is this screen blue?” “Why doesn’t this button do anything?” “Why is it necessary to reboot so often?”

Why, why, why. Good QA people just want to know. They are smart and inquisitive. Their desire to understand the underlying reasons for a feature’s existence is admirable. Good testers don’t just look at the list of features and use it as a checklist.

Excellent testers are unpredictable. They keep asking questions, digging deeper and deeper, until they uncover inconsistencies. World-class testing professionals are like seasoned detectives seeking clues – the biggest difference is that they don’t even know what crime has been committed yet.

One could postulate that software testing is basically about asking questions. That’s what made Columbo one of the best detectives. Is anyone reading this old enough to remember Columbo?

While I cannot give you all the testing answers, I can put forth some good (or humorous) testing questions.

• If I do this, what will happen?
• If I give the system this input, what is the output?
• If I click on these items in this order, what is the resulting behavior?
• You’ve got to be kidding me – did the user actually ask for this?
• Do I like how this flows?
• Does this seem intuitive?
• Is the same result produced every time for the same situation?

  

• Could this complex algorithm fix the BCS bowl/ranking problems?
• How many test cases are enough?
• What has happened in the past?
• Has it always worked this way?
• Why did we change how that worked?
• If we change that function, what else is changed?
• What is the ripple effect of this change?
• How will the customer behave when the software does this?
• How did that function sneak past the project manager?
• Can we predict the outcome for/with these conditions?
• What context is appropriate for the user to understand this feature?

  

• Is this function dependent on anything else?
• How many invalid values do I need to try?
• Does the sequence matter?
• What does the specification say about this?
• Should I invest time analyzing the spec before building test scripts?
• How in the world would anyone be so stupid as to want that feature?
• Who can I ask about the intent of this feature?
• Why are they now referring to Vista as Windows 7?
• Who are the stakeholders?
• How can I try this feature in combination with others?
• Is it necessary to test all permutations of variables X, Y, and Z?

  

• Can this circumstance really occur in normal usage?
• What will our marketing department think about this?
• Did the developer really want the software to do that?
• Will our competitor be envious of this feature?
• Should this test be included in our regression testing?
• What the heck is a “traceability matrix”?
• Am I getting bogged down in one specific area?
• Does this solve a real problem?
• Is this a solution in search of a problem?
• How will my boss feel about this layout?
• Can this color trigger seizures by certain users?
• Is this function in compliance with the regulations?
• Will bloggers say we are crazy for making it work this way?
• Am I understanding the spec correctly?
• Are there any warranties that present legal risk?
• Can we verify the accuracy of the product documentation?
• Does our doc match the reality of our code?
• Have we confirmed that our product can do what is claimed in our marketing brochure?
• Isn’t this a long list?

  

  Long, yes. But not exhaustive. Please share some of your favorite software testing questions. If you have a good detective story involving software engineering or QA, we would love to hear it.