The main goal of quality assurance testing is to make sure that a product is of the highest quality possible before it reaches the end-user. Testers aim to find as many defects as they can in a product before it is handed out to users, and they are there to ensure that the end-user does not experience any major issues when using the product.
Effective testing involves both positive and negative testing to ensure that a product works and behaves as it should.
Positive testing helps to ensure that a product functions as expected, and it allows testers to verify that the system is performing as intended under normal conditions.
So, how does a tester find out if a system is able to cope in unpredictable circumstances? That’s where negative testing comes into it.
Negative testing ensures that an application continues running, even when there’s an error or a user behaves unexpectedly. QA specialists, performing tests in this field, must think outside of the box to understand what could cause the application they are working on to break down.
For example, if the end-user tries entering letters into a numbers only field, they should see a message saying “Invalid data, please enter numbers” popping up on the screen.
The purpose of negative testing is to test the situation above to make sure that the product behaves correctly. Its intended purpose is also to see how errors are displayed and what the user sees if any issues occur, and to prevent application failures by improving its overall quality. Negative testing also allows specialists to find the weak points in a product, and to fix them before the end-user gets to interact with the product.
Furthermore, negative testing is aimed not only at identifying potential flaws, but also at determining the conditions under which something might fail.
Negative testing is often referred to as failure testing.
Examples of where negative testing can be useful:
- The user presses the ‘OK’ button without entering data into text fields - an error message should show up because the fields cannot be left empty.
- The data exceeds the character count - an error message should show up because too many characters have been entered into the box.
- Username contains numeric values - an error message should show up because only letters have been added into the field.
- There is a character in the username field and the application does not allow this - an error message should show up because the field only accepts letters and numbers.
- Invalid words used - an error message should show up.
Negative testing helps QA testers to see how the product behaves when the above cases play out. These are not uncommon problems to encounter, and the product should be able to deal with such issues in the correct manner.
Basic Techniques for Writing Negative Tests
Here are two widely used techniques for writing negative test cases:
- Boundary-value analysis
- Equivalence partitioning
What is “boundary-value analysis”?
Boundary value analysis is when you test the boundaries of a product to see how the product reacts. The values Start-End, Lower-Upper, Maximum-Minimum, Just Inside-Just Outside can be considered as boundary values.
The main idea of this approach is to select the values that are on the extreme ends of the boundary to see how the product holds up in these circumstances. More issues tend to occur near the boundary mark.
Here are some example of boundaries that could be tested:
- Minimum variable
- Slightly above the minimum variable
- Slightly below the maximum variable
- Maximum variable
What is “equivalence partitioning”?
Data, for example numbers, get divided into different equivalence classes, so that you can reduce the amount of test cases that need to be run. Then during testing, one test value is selected from each class.
The idea behind this testing is, that there is no point writing thousands of tests for all valid input numbers, you can just group the data into equivalence data sets and pick one variable from each set.
Tests can be divided into these classes:
- Input class for all valid inputs
- Input data class with all values below the lower limit
- Input data with a value greater than the maximum limit
Challenges of Negative Testing
Not all QA specialists are willing to do negative testing, because they believe that it is a waste of time and energy that could potentially delay the software from being released.
Additionally, positive testing usually gets a lot of attention, whereas negative doesn’t. However, negative testing should not be overlooked as it ensures that the system under test can cope with unexpected functions and conditions.
Negative Testing in Action
Let’s imagine the following… there’s a login screen with two text fields. The first text field requires a ‘username’, the second field requires a ‘password’.
There are specific requirements here. For example, the first field, labelled ‘username’ cannot be left blank (the same requirement applies to the second text field) and the field must contain letters.
The password field, on the other hand, can include any combination of letters (uppercase or lowercase) or numbers from 0 to 9, but no other characters can be used.
The maximum number of characters for both fields is 10.
What does negative testing look like in this case?
- A tester would try to enter a username that consists of numbers and symbols (Examples include: 123 * _) - an error message would show up because there are no letters in the field.
- A tester would create a password that includes special characters and with spaces. (Examples include: \ {^ * &) - an error message would show up because no characters should be included.
- A tester would leave both fields blank - an error message would show up because both fields should be filled.
- A tester would enter more than 10 characters into both text fields - an error message would show up because both fields can only hold a maximum of 10 characters.
The goal of this exercise is to see how the application reacts to unexpected user behavior and unpredictability. To do this, a QA needs to try different scenarios.
Writing negative tests is a process that requires creativity and creative thinking. And testers really need to ‘break’ an application to see what happens.
Testers who perform negative testing do need to keep an eye on the type of testing they do, because there is a risk that the negative testing they perform can turn into positive testing by mistake.
Conclusion
Negative testing is a unique type of testing, whereby the majority of tests are aimed at checking and confirming that a product works as it should during unpredictable scenarios.
Negative testing is concerned with what might go wrong with the system. Therefore, a large proportion of the work done by negative testers is about going beyond the scope of specifications and thinking outside of the box.