Black Box Tools and Techniques
- “Bang on it until it breaks”
- Automated Testing
- Regression Testing
- Boundary Analysis
Bang on the Software
The most common way to do black box testing is to get people other than the development team to use the software. They try things, hopefully planned, which identify defects. These are reported back to the developers for fixing. Although this method of testing is common, it is also common for it to be unstructured without much thought to where defects are likely to be found.
Automated Testing
Automated testing is just the use of a tool to automate some of the testing process – it is not a magic wand for finding every defect. When we think of high-tech tools for testing, automated testing is usually what we think of first. Creating a test case with an automated tool takes more time and expertise than executing a test case manually, but once it is created, it can be run repeatedly until the software is completed. This is not without potential problems, particularly if the user interface is not completely defined, because a change to the UI will likely require automated tests to be modified. Automated test tools include products like LoadStorm.
Regression Testing
Regression testing is where we see the value of automated testing. Regression testing is simply the re-testing of code to be certain that it still works correctly as new features are added and code changes are made. It would not be unusual to have thousands or even tens of thousands of discreet test cases. To execute each of these cases manually might take weeks or months, even with many individual testers. This would be to execute the cases once. If test cases are automated, these tests might be run daily or weekly without many people required.
Programmers can make more errors modifying existing code than creating new code!
Boundary Analysis
Boundary analysis is the testing of limit conditions. Test cases that generate program values that are at or beyond the program’s defined limits are created for boundary analysis. This is commonly an area where programmers make mistakes.
White Box Tools and Techniques
- Code Review or Walkthrough
- Compile Without Error
- Unit Testing
- Path Analysis
- Code Coverage Analysis
- Profiling
Code Reviews
Code reviews are just a meeting to look over one person’s code by one or more peers. They can be done formally or informally. Although “low tech”, code reviews have proven to be effective in finding defects early and improving code quality.
Compile
Many development environments have adjustable warning levels and add-on products are available for formatting and analysis of potential problems. Tools include the compiler for compiled languages and Lint from Gimpel Software.
Unit Testing
Unit testing is usually done by writing code, sometimes call scaffolding, to test a particular unit or piece of code. the “scaffolding” code is usually written in the same language as the unit is written in and is often removed when the test is done.
Path Analysis
Path analysis is tracing potential execution paths through the software. This is done on static (not running) code. Path analysis will identify particularly complex functions and interrelationships between functions and modules. The more complex functions will likely have higher numbers of defects. These code areas may need to be rewritten to make them simpler or have a higher concentration of testing. Tools such as those from McCabe and Associates are used for path analysis.
Code Coverage and Profiling
Code coverage analysis and profile tools work on dynamic (running) code. These tools help to identify which lines of code have been executed, how long the execution took and how often these lines are executed for a particular test. Some development environments have tools for coverage analysis and profiling included. Many compilers have switches for profiling where output can be dumped to a trace window.