I’d argue that the most underrated type of testing in software engineering is exploratory testing. Some people assume that it’s a task with low-effort thinking, where the tester simply goes through the application and sees what comes up.
We all know that exploratory testing is not ad hoc testing, which implies that there are techniques and patterns that need to be followed. I use a two-step process when I conduct exploratory testing on a software application that has undergone a change.
Step 1: Approach the application under test without the knowledge of what has changed.
This will ensure that existing behavior is not affected and critical-path functionality is not altered. If your test fails, it will unravel the changes the application has gone through, which could either be expected behavior or a potential defect that was introduced while implementing the change.
In order to do this, though, a critical-path product plan needs to exist so I can refer to it each time a change is made. And this product plan needs to be up to date and maintained by the entire team.
Step 2: Approach the application under test after gaining complete knowledge of the expected behavior.
This would require a thorough analysis of the user requirements document. The tester needs to work closely with the engineering and product team members to understand the areas of impact, which would in turn help uncover edge cases. This also requires a complete understanding of the end-to-end product behavior, most importantly integration points.
Let’s look at an example to better understand the importance of exploratory testing and how the above two techniques can be beneficial.
Assume we have a software application that lets users upload documents, and it’s implemented using a multi-layered architecture:
- Presentation layer—the GUI that helps users upload, replace, delete, and download documents to and from a folder on the server
- Application layer—runs the business logic, like validating file types, verifying user permissions, etc.
- Data layer—stores the uploaded files, audit entries, etc.
The application currently only allows pdf files to be uploaded by users with a certain role, A. The change requested by the product team is that users with role A would no longer be allowed to download files and that a new role would be added, role B, which can upload jpg and png files along with pdf files.
After this change is implemented, it’s up to the tester to do exploratory testing. In step 1, without the knowledge of what has changed—the ability to upload jpg or png files—the tester would initially try uploading pdf files, replacing them with another pdf file, deleting, and downloading the uploaded pdf file as a user with role A. So, this test would fail when the tester tries to download the pdf file, as that ability is removed by this change for that role. The implemented change should not break any other flow.
This approach will make sure that existing critical-path functionality is not impacted and also reveals that there has been a change that does not allow the tester to move ahead, prompting them to move on to step 2.
Now, the tester goes through the user requirements document and analyzes the changes required. Areas of impact would include all three layers of the architecture, so tests would be devised for each layer:
- Presentation layer—error message verbiage, ease of operations, etc.
- Application layer—file validations and permissions-related tests
- Data layer—file size and audit entries
By now you may be thinking that these are normally the steps taken by every tester while testing an application. Where does exploratory testing come into picture?
Let’s say that while following the above steps, you take a screenshot, which gets saved as a jpg with the current date and timestamp as part of the filename. It has special characters and spaces due to the date and time format, and when this file is uploaded, it would fail the test, as the space in the filename would get converted to “%20” and the code might not be able to handle it.
This fact would not have been mentioned in the test plan or as acceptance criteria; it can be uncovered only by exploring the application with different perspectives.
When you try to solve a problem in your own way, you tend to discover a whole new perspective. Someone else may have tested the process with any image file, but because a tester used a screenshot, it uncovered the issue.
The availability of the file might have been a critical requirement for a downstream application. This issue could also have been discovered during integration testing, which serves as another important test to keep in mind while conducting exploratory testing. Making sure all integration points are kept in mind and that the right data is pulled from or pushed into these points is very important.
Another test that could be done in this case is load testing. The application should also handle concurrent users trying to download the same file. An exploratory tester should also have the ability to picture the application in a real-life scenario.
Exploratory testing is all about thinking with different mindsets, as different users, and going over all layers of the application while thinking of all possibilities for use cases.
There are several tools that can be useful in following this technique. I have always been a big fan of making mind maps to represent the end-to-end functionality of the application. It helps me to find areas of impact seamlessly and to avoid getting bogged down by intricacies and losing sight of the big picture.
Also, exploratory testing does not always have to be associated with manual testing. The results of exploratory testing could uncover areas needing automated tests.
In order to perform exploratory testing efficiently, it’s important that every member of the team follows a set testing process. Everyone needs to wear the hat of a tester and do exploratory testing at some point in the software development lifecycle, starting from requirements gathering all the way until changes are released to production.
Exploratory testing is all about adding perspective to product, process and people.
Originally published on StickyMinds.