In 1984 Cem Kaner coined the phrase Exploratory Testing and used an interesting analogy in a 2008 talk to compare the differences between it and automated/scripted testing before fully defining out exactly what it is. Before diving into exactly what exploratory testing is, imagine for a moment that you are a police crime scene investigator who has been assigned a murder to investigate. But for this investigation you are specifically restricted to a predefined set of scripted actions you can take to get to the bottom of the crime. You cannot deviate from this script and thus are severely hamstrung in solving the murder unless the outcomes of the scripted actions and questions lineup. This is exactly what scripted/automated testing does. It hamstrings the test process only looking for a predefined set of designed questions and thus, the need for exploratory testing.
Exploratory testing “emphasises personal freedom and responsibility” in testing because the individual tester tests, learns and iterates throughout not looking for a pre-designed answer that cannot deviate. This is not meant to mean it is not lacking in preparation but rather not constraining the tester. The tester is free to explore and even encourages creativity toward problem solving. The three stages below are not independent of one another and exist as a single, cohesive function or test cycle.
One of the most important functions that a tester needs is an understanding of the app or website that they are testing. This understanding provides context and includes information such as competitive benchmark data, industry knowledge and company details. An understanding like this ensures that the tester is able to take in all manner of inputs relating to the app or website when he or she is performing the actual test. Contextual knowledge allows testers to provide details surrounding results that they may find and whether or not they are applicable.
They also need to understand deeply the requirements of the specific test that was requested. A deep and comprehensive understanding of the testing requirements arms the tester with the right amount of knowledge to dive deep into the app or website. When the tester knows the requirements of the testing, he or she can make sure to keep their testing and related bugs within those functional realms.
It is also important that the tester knows what the desired outcomes of the testing might be. It is not enough to merely look for bugs but to know what the end goal is for the testing cycle. Sometimes the goal is to ensure that previous bugs have been resolved in such a way that new functional bugs have not been introduced (regression) somewhere else. Sometimes the software testing is to ensure that particular functions work as expected and no bugs found are the desired outcome per se.
A major difference between exploratory and scripted testing is in design. Designing differs from scripting in that while the test has specific parameters or rules, it is not done in a preset path or prescribed manner. Exploratory testers are able to conduct the test in a way which they deem fit and do not have a desired or expected outcome. Often times, the exploratory testing technique leads to developing more rigorous, scripted test scenarios over time.
Designing also affords testers and test managers the option to map out the various techniques a tester might use. This could mean the device, circumstances or conditions if that has not been established yet by the test requester. Traditionally however, testers and test teams map out the timeboxing of the test, the number of testers needed and other important pieces of the test cycle. The test cycle still is not a formal test case base nor is the tester writing out test cases during the testing itself. Testers can use notes, mindmaps, flowcharts, decision tables or any manner of organisational tools at their disposal.
Finally, the tester is given the freedom to complete the test as he or she feels free to do so. As soon as the test is written or requested (note: not a test case), the test can be conducted. This freedom means that nobody is waiting for scripted requirements and creative work can begin. A tester then is observing and learning about the application or website. They are probing and exploring how functions work, how they interact with one another and how those pieces work together so that further exploration can take place. Results are then compiled and reported back through the appropriate methods.
Exploratory testing is a form of testing that encourages and rewards both the product designed and tester by allowing an unconstricted approach to finding bugs. This is stark difference between exploratory testing and adhoc testing although frequently the two are construed to mean the same thing. Adhoc approaches are distinguished by their lack of defined process and approaches.
When you think about think test approach, it can be easy to think that the test methods rely on intuition or gut. However, this is not the case. Intuition and gut play a role during when a tester goes to test the application or software but in the test plan there will be specific goals, functional areas and areas in which functionality is assessed.
It is best to imagine the functional exploratory tester as a detective who is investigating a crime. The crime has been committed (expected functionality) and the detective is working his or her way through the clues to determine whether or not everything makes sense. Providing this level of freedom to assess your products and services ensures you deliver a level of quality your consumers absolutely demand.
Exploratory Testing Example
A client of Global App Testing uses exploratory testing in a very popular iOS and Android mobile gaming app. When this client is ready to release a new version of their application, usually on a bi-monthly basis, they call upon exploratory testers from the global crowd to functionally explore the app.Testers from around the world will be given general scenarios from which to test expected behaviour of the new version.
For instance, the end user might expect to be able to play against another human or computer player. In doing so, they might need to use tokens or in-game inventory items before battle. The exploratory testing may require a tester to ensure that this functionality works and the player can complete the battle as expected.
These kinds of tests can’t really be executed via automation. While a script could ensure that tokens or inventory items are purchasable, it can’t emulate a human doing battle! And this is the inherent human aspect of exploratory testing.
When does exploratory testing fall short?
Exploratory testing doesn’t always provide the most efficient use of time however and there are situations where test case execution and regression testing make more sense. In the same example as mentioned above, if the company releases a new functionality that is unrelated to the in-game inventory or currency, it makes little sense to use testers in a functional way. Simple regression tests can be conducted to ensure that the other features have not been inadvertently affected by the new features released elsewhere.
It is important that a good test plan is put in place at your organisation so that you can be efficient and use time wisely as release deadlines approach. We often run into companies who cut short their testing time in cycles because they have simply run out of time before release. However, with a good plan in place (and using the right scalable solutions) you can avoid these kinds of issues.
What type of testing does your organisation need? Are you developing a mobile application for consumer user? Or, does your company focus on B2B applications and thus is focused on delivering API functions? Whatever the products or services you offer, different types of testing are required at different stages. When parts of your products and services are stable and only exhibit small changes, you might only need to consider regression testing. However, new functionality and changes might demand you utilise exploratory testin to ensure all the bugs are found before release.
As part of your organisations testing, exploratory testing and test case creation/execution can be executed through the use of a professional crowdsourced QA team. Leveraging the experience of experts, your organisation can access decades of testing experience designed to provide you with continuous testing strategies that speed up your development process. Contact us to learn more.