The Foundations of Testing

 

What is the ultimate goal of testing and quality assurance?

It is to increase the quality and reliability of a system. A system has quality when it functions as the client or user expects. It is reliable when it can be executed for a period of time without failing.

What is the difference between testing, quality assurance, quality control, verification and validation?

Testing is an organized process of identifying discrepancies, i.e. the variances between the actual versus the expected results.

Definitions of quality assurance, quality control, verification and validation differ from one organization to another. It is probably not worth looking for agreement on these terms. Just remember these concepts instead:

A system must do the right things and do things right.

System quality and reliability can be improved both through inspection and execution.

System quality and reliability can be improved both by “insiders” (the development team) and “outsiders” (The QA and testing team).

What are the building blocks for ensuring quality and reliability?

People — The most important component of testing and QA is the team. This team includes people with testing and quality assurance expertise, users, clients, stakeholders, partners, and marketing people. An inclusive team will ensure that the required skills and attitude are present.

Documented requirements and design specifications — the design specifications make structural testing possible. Not knowing what a system is supposed to do makes it difficult to know if it is actually responding correctly and efficiently.

A stable target — there needs be a certain amount of stability in the requirements and specifications, especially if test planning starts early in the project. It is very difficult to plan or to execute a test when the target is moving.

Management support — there must be top-level management recognition and support for any testing and QA initiative.

Risk and exposure analysis —the functionality of a system should be analyzed for risk factors. These can be based on the importance of the system, the probability of failure and the likelihood of easy detection. But remember, it is impossible to test useful software exhaustively.

Standards — applying accepted standards to the entire system life cycle, including both process and deliverables is key. Without standards, it will be difficult to measure quality.

Plans — if test plans are developed and refined at the same time that the analysis, design and development phases are being implemented, they can serve to provide a “second pair of eyes” on the process and deliverables, as well as guaranteeing that testing has fidelity. In other words, the test design will more likely represent the real world use of the product.

Tools — testers need tools to extend their capacity to perform complicated tasks. These tools may range from spreadsheets of test cases to databases for managing software and hardware configurations, and include especially developed automated test tools.

Coverage, coverage, coverage! — more than anything else, this is the most important building block of testing. Testers need to ensure that they use methods that systematically generate test cases that can cover testing of the global, the functional and the nonfunctional requirements of the system.

What are 3 common testing strategies?

Delay testing and test planning until after development. This is the least desirable.

Plan the tests early, but still delay testing until after development. This is marginally better.

Structure the process of analysis, design and development so that there are concrete interim deliverables at each stage or mini-stage. Inspect and check these deliverables before proceeding to the next step to help ensure quality from the start. This is the best approach.

What are some common testing dilemmas?

Can’t let go?	Balance the desire for “certitude” (making sure that you can reproduce a failure) with the need to make progress on the testing as a whole. This is one part of the overall dilemma involving pressures to implement the product with pressures to continue testing and fix defects.
Whose work is it?	Avoid crossing the line from “isolating” defects to debugging them.
How much testing?	Testing too often uses too many resources, not testing often enough makes it difficult to isolate regression from improvement.
How can testers be efficient?	Determine the methodology that will get the maximum coverage for the fewest test cases.
Developer addiction?	Try to avoid developer addiction to the test team by making sure developers still take responsibility for unit testing.
Black or white box testing?	Choosing the balance between black and white box* testing will depend on the quality of the specifications and the ability of the testers to understand the code. *Black box testing = testing only according to specifications. White box testing = testing with knowledge of underlying program code.