Software Testing
Software testing is a process used to help identify the correctness, completeness and quality of developed computer software. With that in mind, testing can never completely establish the correctness of computer software. Only the process of formal verification can prove that there are no defects.
There are many approaches to software testing, but effective testing of complex products is essentially a process of investigation, not merely a matter of creating and following rote procedure. One definition of testing is “the process of questioning a product in order to evaluate it,” where the “questions” are things the tester tries to do with the product, and the product answers with its behavior in reaction to the probing of the tester. Although most of the intellectual processes of testing are nearly identical to that of review or inspection, the word testing is connoted to mean the dynamic analysis of the product– putting the product through its paces.
The quality of the application can and normally does vary widely from system to system but some of the common quality attributes include reliability, stability, portability, maintainability and usability. Refer to the ISO standard ISO 9126 for a more complete list of attributes and criteria.
Alpha testing
In software development, testing is usually required before release to the general public. In-house developers often test the software in what is known as ‘ALPHA’ testing which is often performed under a debugger or with hardware-assisted debugging to catch bugs quickly.
It can then be handed over to testing staff for additional inspection in an environment similar to how it was intended to be used. This technique is known as black box testing. This is often known as the second stage of alpha testing.
Beta testing
Following that, limited public tests known as beta-versions are often released to groups of people so that further testing can ensure the product has few faults or bugs. Sometimes, beta-versions are made available to the open public to increase the feedback field to a maximal number of future users.
Gamma testing is a little-known informal phrase that refers derisively to the release of “buggy” (defect-ridden) products. It is not a term of art among testers, but rather an example of referential humor. Cynics have referred to all software releases as “gamma testing” since defects are found in almost all commercial, commodity and publicly available software eventually. (Some classes of embedded, and highly specialized process control software are tested far more thoroughly and subjected to other forms of rigorous software quality assurance; particularly those that control “life critical” equipment where a failure can result in injury or death). (see Ivars Peterson’s Fatal Defect for counter examples).
White-box and black-box testing
In the terminology of testing professionals (software and some hardware) the phrases “white box” and “black box” testing refer to whether the test case developer has access to the source code of the software under test, and whether the testing is done through (simulated) user interfaces or through the application programming interfaces either exposed by (published) or internal to the target.
In white box testing the test developer has access to the source code and can write code which links into the libraries which are linked into the target software. This is typical of unit tests, which only test parts of a software system. They ensure that components used in the construction are functional and robust to some degree.
In black box testing the test engineer only accesses the software through the same interfaces that the customer or user would, or possibly through remotely controllable, automation interfaces that connect another computer or another process into the target of the test. For example a test harness might push virtual keystrokes and mouse or other pointer operations into a program through any inter-process communications mechanism, with the assurance that these events are routed through the same code paths as real keystrokes and mouse clicks.
Where “alpha” and “beta” refer to stages of before release (and also implicitly on the size of the testing community, and the constraints on the testing methods), white box and black box refer to the ways in which the tester accesses the target.
Beta testing is generally constrained to black box techniques (though a core of test engineers are likely to continue with white box testing in parallel to the beta tests). Thus the term “beta test” can refer to the stage of the software (closer to release than being “in alpha”) or it can refer to the particular group and process being done at that stage. So a tester might be continuing to work in white box testing while the software is “in beta” (a stage) but he or she would then not be part of “the beta test” (group/activity).
