Just some testing terminology…

In this post I will try to make clearer several terms used in testing methodology and also to answer some questions that could generate confusions for many junior level testers. I won’t make a testing terms glossary but I will just go through some misunderstandings which I also faced myself some time ago.

I will start by tracing the difference between verification and validation procedures. Generally the answer for this is that

verification is when you check to see if the product was built right and validation is when you check for the right product.

Other definitions would be verification is truth and validation is correctness. This subtle difference has a lot to do with the difference between smoke testing and sanity testing, but I will come back on this topic later on this post.

One meaningful definition that I’ve found and I would like to share it with you is the one in the context of V software development model.

As picture above shows it, in the V model of the software development process, design and testing takes place at the same and one is influenced by the other. Development team is always triggered by the inputs that it receives from testing team. Development is the left hand side of the V and testing is the right hand side.

At the beginning user requirements are analyzed in order to decide how the system will look like in general. How the big picture will be. Based on that a set of user acceptance tests are executed (by the way acceptance tests are among first tests executed on a product) in order to see if the product was built accordingly, if it makes sense to further into details with the implementation.

The outcome of this step should be something like this:

We need to use those modules, those menus and structures

Next thing would be to establish the functionality of the whole system, this is usually referred as being the high-level design.

How will work together those modules? Which interfaces need to be used and what types of structures, menus or modules will be needed?

Those are topics belonging to this development stage. In parallel the testing team has to prepare the system tests, their output is used by the development team to decide whether or not they can pass to next stage.

As you may have noticed left hand side of the V, besides the fact that is development-related, consists in some preliminary verifications and then, after the implementation is done, product is sent for validation. No test code is executed in the verification phase, it consists only in analyzing documents and requirements. The effective testing activities take place in the validation.

… but let’s go further with the V …

After the high-level design is tested and the feedback is OK, the project passes into its next and last stage, low-level design and testing. The high level system is broken into smaller units and the implementation is done at the most detailed level that the project supports. After this design phase the test team has to check if they made a good work, unit tests are ran at this stage. When this stage is successfully completed, integration tests are done in order to verify if implemented units work accordingly when connected together.

…this was it with the V model and Verification and Validation ….

Another interesting thing to know is the difference between smoke and sanity tests. Briefly sanity checks are to see whether the validation worth continuing and smoke tests are to see whether validation can continue. Smoke tests are to see if the deliverable has is stable enough to be tested further and on the other hand sanity tests usually are applied after some minor updates were applied to the deliverable in order to fix some issues. Before starting again the full regression testsuite it is worthwhile to verify if corresponding issues have been fixed. This is what sanity tests should check for.

As the names implies: smoke tests are to see if the product won’t catch fire if some basic functionalities are turned on. They are usually applied to a large set of product functionalities

Sanity tests are done in order to see that the product is healthy again after few of its functionalities were damaged. They are applied to a narrow area of functionalities.

Another important thing to have in mind is the difference between integration tests and system tests. The main difference between them consists in the moment in time where those are applied. Integration tests are applied before system tests. The purpose of the integration tests is to verify if modules linked together still work fine and are executed after completion of unit tests. Integration tests check only for the connections between component parts and their interfaces. System tests usually are the last stage in testing procedure. The functionality of the entire application, considered to be a black-box, is tested at this level, which is not the case for integration tests. Stress tests, volume tests, regression tests, they all can be part of system tests.

And before ending just to clarify another thing. What’s the difference between stress and volume testing?  One important thing that differentiates them is the quantity of what is called test vectors. Stress testing does not necessarily refer to a large volume of input values applied to the system, even if this may be one extreme condition which fits for stress testing. Stress testing refers to check for the functionality of the system under extreme or critical conditions. It may be the following scenario: when testing a microcontroller, the supply voltage is lowered until a threshold is reached, simulating a power-off or a black-out voltage, or the PLL is configured in such a manner that its frequency is close to the maximum/minimum allowed value . Those fit for a stress testing because some critical conditions are applied to the microcontroller, but it doesn’t fit for volume testing because the test is not about how can the system handle multiple transactions in a limited time, or something like this.

Volume testing, as the name implies, is related to the quantity of input data applied to the system.  The system is checked against large volume of data that it can process.

Those two along with load testing are categorized under performance testing.

A very good resource that I kindly recommend you to consult if you want to enhance your testing expertise is this one .