One of the most important roles on a software development project team is Testing and Quality Assurance. This role on the team is charged with verifying that the application or system complies with the functional requirements. Or put another way, this role finds the bugs.
A testing team will perform many tasks in order to achieve their objectives. The testing personnel should be actively involved in the development process to ensure that quality is built into the product from the outset. It is not optimal for quality to be “tested into” the software. Therefore, it is most effective when the quality assurance team is involved from the very beginning of a project. They can help greatly in avoiding mistakes and preventing defects from being coded.
Prevention is integral to the software development process, but it is not part of testing in the strictest sense. Rather, prevention is part of a broader topic of quality assurance. This article will focus on the process of detecting defects, and preventing defects is best done by defining requirements accurately and completely.
If there are no clear specifications, we are very limited in determining whether an application works properly or not. If the developer is supposed to “just know” when it works, or if working is deined as “not crashing”, then one of the earlier phases of software development has been skipped or done poorly.
Does this mean that as the developer “it’s not our fault” when something goes wrong? It should not be a subjective decision whether software fails or not. We need to define what the software does before we build it. If we accept bad requirements, it IS our fault. If we build software based on little more than ideas, then we are building a prototype, and it should be defined as such.
What is a simple definition of testing? The detection of defects.
What is a defect? A failure to meet specifications or perform as a user expects.
Black and White
Black box testing means that the tester can only see and test the application through the user interface. The implementation (code) is hidden (inside the black box) and we cannot see inside. Blac box testing is typically done by someone other than the software developer such as a professional tester on the development team or a beta test client. Black box testing is usually done later in the life cycle, after the code has been written.
White box testing means that the tester can see the code. It is inside a white (or glass) box that we can see through. White box testing is typically done by the software developer or a peer. White box testing is usually done earlier in the life cycle, while the code is being written.
Phases of Testing
- Unit Test
- Integration Test
- System Test
- Beta Test
- Acceptance Test
Unit testing is testing of a function, feature, module, object or other small piece of code that is separate from the rest of the software system. It is generally a piece of code that one person has developed. The purpose of unit testing is to verify that the unit, by itself, functions correctly.
Integration testing is verifying that the individual units work well together. This area of testing is often fuzzy and not well understood. The goal here is to find and fix any problems with the code or data sharing among the units. Integration tests may uncover undesirable “ripple effects” between units.
System testing refers to testing of the whole system after it has been integrated. Because system testing falls within the scope of black box testing, it does not require knowledge of the inner workings of the code. Less commonly, system testing can be conducted on an incomplete system where most of the units have been integrated and the overall application can perform a discreet set of functionality.
Beta testing is done by releasing an unfinished product to a limited population of the market for them to work with. Beta testers generally get the first public look at the product in exchange for reporting of problems in the software. Beta testers are normally unpaid and not associated with the company producing the software. In many cases, beta testers will get priority in defining feature priorities for the final release.
Acceptance testing is normally done when software is being produced by a development company for delivery to another company. For example, contract developers write the code to a client’s specifications, and the client puts the application through acceptance testing to verify that they are getting what they expected. This is the final testing that is done to prove that the software meets all of the contractual requirements, and therefore, can be accepted. In such a scenario, the last payment is tied to the acceptance. Many companies use acceptance testing internally between their user community and the IT department in the same manner as a outside contract developer. It is a good way to hold the internal development team accountable to meeting expectations.
Load Testing or Volume Testing
When running a simple program that calculates a difficult numerical analysis (e.g. MS Excel), the speed of execution is normally bound by the CPU. A desktop database query (e.g. MS Access) is normally I/O bound. On client/server or web applications that run on two or more machines, it may not be clear where the bottleneck exists. This adds another dimension to testing because we need to test conditions that will not easily be simulated by testers. To prove that an app works with hundreds or thousands of simultaneous connections you need more than the testing team of a few people – unless you have special tools.
An experienced tester can create automated tests that will simulate the behavior of many users. Using load testing tools, thousands of virtual users can be generated to concurrently operate in the system being tested.
Load tests show how the software performs at its intended scalability limits. These limits may be number of simultaneous users, queries, or complex transactions, or a combination of these. We would expect some performance degradation of response times as the system reaches its limits. The requirements should specify limits and acceptable performance parameters. We also should expect graceful handling as these limits are exceeded.
Stress Testing
Stress testing is related to load testing, but there is a different objective. Stress tests are designed to break the system. Some examples of ways to stress the application are:
- Too much volume
- Hard drive failure
- Low memory condition
- Application server crash
- Load balancer fault
Most people think of stress tests as excessive volume, but it can be very beneficial to stress the system in other ways. For example, in your storage array you can pull out one of the RAID drives to simulate a drive crash. Place a new drive in the array and see if all mirrored data is correctly reproduced, and more importantly, what happens to the system performance? By finding out during a stress test, you have the opportunity to correct any configuration problems. Stress testing can help assure that customers don’t notice or they see minimal impact when something bad happens.
To be continued…