A popular question when studying testing features is the difference between testing methods. Finding information about Black-box and White-box is not difficult, but Gray-box is not popular among authors of educational articles. We decided to fix this and help you figure out how the Gray Box test differs from others.
Often, the Gray Box is considered a combination of White/Black-Box types, since it implies that we only partially know the internal structure of the tested product. Therefore, before trying to understand what Gray-Box testing is, it is worth understanding what other methods it consists of.
Black box testing
When testing using the Black Box method, the tester does not have access to the internal structure of the system components. Therefore, the procedure for obtaining and selecting test cases is based on the analysis of the specification of the system components without direct knowledge of their internal structure.
- The simplest example of Black-Box testing would be any test for a notification trigger, when sending functionality is affected during testing, and the tester does not have access to mailboxes / database.
- With this strategy, the tester checks the product without having information about the features of its implementation, using only the interface provided by the developer.
- In this case, the Expected Result will be covered by the Requirements and/or Specification.
White box testing
White box testing involves working with an “open” system, where its internal structure, as well as the device and implementation are known in advance to the tester at the time of the start of the tests.
The tester has access to the implemented code, test documentation, studies them and receives all the necessary information on how the product should work.
At this stage, it remains only to check the actual results of the implementation, compare them with the expected: certain code and project documentation.
- Any open source project is a good example.
- By downloading and running similar ones, you can write autotests, the run of which will become a test. Such projects often lack a user interface, which cuts off the possibility of testing Black-box.
Main Differences Between Black-box and White-Box
White box testing requires knowledge of programming. Therefore, it is believed that developers themselves use this type, since they know the code. They define appropriate or inappropriate design patterns, class structures. For them, the system they are developing is the White Box.
Black-box does not require programming knowledge, so the Testing department works directly with it.
Gray box testing is guided not only by the specification, but also by key design elements.
Provided information about the database. If the program uses any database for its work, we can analyze the types of fields in which program variables are written. And then analyze the restrictions that the base imposes.
For example, if the user’s last name being entered is in a 128-character string field, we would:
- try to find a surname longer than 128 characters – there will be a rather serious bug, if such surnames exist – a person with such a surname will not be able to use our system;
- regardless of whether such surnames exist or not, try to enter a string of more than 128 characters – the program should not break (a clear error message should be displayed).
If the program integrates with other external systems besides the database, the limitations of such systems can also be analyzed. For example, if we are testing an IMAP mail client, we should make sure that it correctly handles long paths to folders on the server (most often, the path length limit is 255 characters).
In addition to the above, if we also have access to the program code, then we can:
- pay attention to special cases that were not included in the requirements document and which need to be tested;
- on the contrary, to see that it makes no sense to test some things.
Advantages of the method:
- Gray box testing includes the benefits of black box and white box testing. In other words, the tester looks at the test object from the position of a “black” box, but at the same time conducts analysis based on data about the system that he knows.
- The tester can design and use more complex test scenarios.
- The tester works together with the developer, which allows you to remove redundant test cases at the initial stage. This reduces the time for functional and non-functional testing and has a positive effect on the overall quality of the product.
Disadvantages of the method:
- The ability to analyze the code and test coverage is limited, since either there is no access to the source code, or the tester does not have the knowledge to study this code.
- Tests can be redundant when the developer also checks their code with unit tests.
- You cannot test all possible input and output streams because it takes too long.
The strategy combines the goals of the other two boxes, at the same time it is a universal option for cases where it is not possible to resort to White-Box testing, but there is a need for more significant test coverage than Black-box can provide us. Gray box testing will be closer to the Black box due to the lack of need for the tester to have access to the source code. All tests are created based on knowledge of the algorithm, architecture, internal states, as well as other high-level descriptions of the program’s behavior. .