What is Exploratory Testing? | Global App Testing

What is Exploratory Testing? | Global App Testing

This article covers “exploratory testing” – a kind of testing which forgoes a script or test cases, and instead invites testers to find bugs however they can. As a supplier of crowdsourced exploratory tests, you can get started with Global App Testing to learn more! 

Want to drive your global growth in local markets?

We can help you drive global growth, better accessibility and better product quality at every level. 

Chat to an expert

What is Exploratory Testing?

Exploratory testing definition

image1-Jul-20-2023-08-18-54-6495-AM

“Exploratory testing” is the practice of allowing testers to enter a product and find bugs and errors without the help of a script or test cases. It encourages testers to think creatively and simulate real-world usage, mimicking the actions of end users. In the case of Global App Testing, testers are incentivized to find bugs which the QA team considers high or critical priority.

Exploratory testing increases the robustness of the overall QA portfolio, finding edge case tests which exist outside of scripts for greater overall quality.

Is exploratory testing always functional testing? 

Like many definitions in QA, we’ve seen “exploratory testing” used in multiple ways. Usually the term does refer exclusively to functional testing. In this article we’ll use “exploratory test” to mean “exploratory functional test”; but within the Global App Testing product, you can ask our testers to focus on multiple kinds of issues including functional issues, localization issues, usability issues. We call this an “audit” (our launchpad doesn’t refer to it as “exploratory testing” for this reason) and you can easily select this as you go to launch a test.

Why do businesses undertake exploratory tests?

The term “exploratory testing” was coined in 1984 by Cem Kaner (a well-known software engineer and academic) to differentiate the practice from scripted testing using test cases. Exploratory tests are often used to complement a set of test cases and provide greater robustness.

The weakness of test cases as a mechanism for finding bugs is that by following a predefined script for where to look for bugs, where the script misses bugs, those issues will compound and get worse as the script is reused again and again. Adding the element of randomness to the system prevents systematic mistakes in your QA process as well as providing deeper overall coverage.

Another advantage of exploratory testing is that it takes advantage of the expertise that testers have accrued by testing a lot of software. In Global App Testing’s “crowd” of 90,000 testers, we incentivize testers based on performance in order to deliver our clients the best possible test execution. Exploratory tests are often “better value” than test cases in the sense that they sometimes deliver more high or critical importance bugs at a greater acceptance rate than test cases per credit. 

Exploratory testing suits this much better; it encourages testers to use their expertise to deliver critical bugs quickly in addition to the clear documentation and communication we’d expect anyway in the delivery of test cases.

When is exploratory testing done?

All product and QA decisions ultimately derive from the needs of the business. Exploratory tests are a measure we undertake to drive robustness across your QA. So you might undertake them if:

  1. Your critical test cases have been tested and you are not conducting exploratory tests as a substitute for a legal or safety absolute requirements 

  2. Exploratory testing seems to be more effective in delivering bugs than other kinds of test, e.g. an exploratory test seems to be delivering more high or critical importance bugs than test cases for your busines

  3. You are dependent on repeat test cases, and you are concerned about the robustness of those test cases in identifying every bug your product might have 

  4. You want to take a look at “edge cases” in your software. For example, individuals with the Korean surname Le often fall foul of forms which require a minimum three-letter surname. This is an “edge case”, where the form works but it is possible to create a scenario in which the form is broken. Exploratory tests are great for identifying these 

  5. You do not have test cases written, and you want to test your product without test cases (please note that GAT can write test cases for you, if you would like).

We don’t recommend exploratory testing as a totalized test strategy. It’s a great way to complement test cases, and we’d say you really need both – the first to ensure you have coverage planned for your whole system, and the second to add extra robustness to that process.

Example of an exploratory test 

ConnectX, a (fictional) social media platform, decided to undertake an exploratory test to ensure the quality and reliability of their posting functionality. ConnectX scheduled the exploratory test during the development phase of a major app update to ensure that the test cases across regression and new feature testing had comprehensive coverage.

The assigned tester conducted the exploratory test by logging into her test account and navigating to the home/feed page. She explored the options available for creating a new post, testing various combinations of text and media content. She paid attention to how the app handled different scenarios, such as uploading images, videos, or links. Additionally, she tested the privacy settings, verifying that they were applied correctly.

The tester reported several bugs, including:

  • The instances where uploaded images failed to display; on android phones holding the phone horizontally. 
  • Where the privacy settings were not properly applied; because you could see the private email of other users when you hit “inspect” on one of their feed posts 

ConnectX was able to address the issues before releasing the app update to their users. This resulted in an improved user experience, as the identified bugs would. The above tests found valuable bugs which did not exist inside the test cases and would not have been surfaced in a test-case only approach.

Exploratory Testing: How it works

image2-Jul-20-2023-08-18-54-7277-AM

Exploratory testing follows similar steps to a regular test:

1/ Product learning 

Specialized and B2B applications may require some education before an exploratory test can be effective. With Global App Testing, this can either involve teaching (i.e. using our materials and framework acquired during our 100-day onboarding but educating testers to a minimum standard) or a play period in the app to help testers familiarize themselves with a product; often both. 

2/ Outcome learning 

Testers should also know the desired outcomes, whether it's identifying flaws in early iterations or ensuring resolved bugs don’t keep being reentered. Again, Global App Testing provides testers with requisite materials and knowledge to minimize noise in their bug delivery.

3/ Execution

Testers have the freedom to conduct tests as soon as they are written or requested. They observe, probe, and explore the application or website, compiling and reporting results through appropriate methods. This approach encourages an unconstrained search for bugs and rewards both the product and the tester.

4/ Reiterate

Each round of exploratory testing provides new data for the incentivized testers as you accept or reject the bugs and tag their severity. This means that as you keep undertaking exploratory tests, your exploratory tests should become more effective at identifying issues in your software.

 

Exploratory testing within agile 

Find our longer blog on exploratory testing within agile teams here

Exploratory testing is difficult to automate, which makes agile development with an exploratory test embedded into the sprint cycle a challenge. Global App Testing can provide a 48-hour turnaround for exploratory tests (with the exception of very large test – Ts & Cs, etcetera) using a crowd, so this isn’t a problem with us – but can pose a general challenge for businesses. 

Exploratory testing tools and services

image3-Jul-20-2023-08-18-55-8906-AM

Chat with a QA expert

Exploratory Testing service: Global App Testing 

Global App Testing is a provider of crowdtesting services and we’re extremely well placed to offer exploratory tests for the deepest possible quality checks.

Advantages of crowdtesting for your exploratory tests

Crowdtesting is a process of testing using a “crowd” of professional testers around the world to test your product.. In the case of Global App Testing, we have over 90,000 testers available at the time of writing (July 2023) and we expect our crowd to keep growing as we do. 

Crowdtesting tend to marry the advantages you might associate with specialized testing service providers with the advantages you might associate with automated tests.

Those advantages are:

  • High availability of devices and labor. This takes much of the hassle involved with device management out of certain kinds of tests (such as version testing) and provides you with a great cross-section of devices, versions, and if necessary, demographic and geographic locations of testers.

  • Multi-time zone and high tester supply, meaning that you can launch any number of tests at any time and get it back quickly. This helps us fit our tests into agile sprints with overnight or weekend testing. Crowdtests are slower than automated tests but much faster than other kinds of manual testing, which confers a huge advantage.

Advantages of Global App Testing as an exploratory testing service

Global App Testing offers several unique advantages over other crowdtesting providers. 

As the oldest crowdtesting provider we’ve been around the block, and we’re very lucky to have been trusted by some of the biggest names in technology. Google, Meta, Microsoft and Atlassian all trust Global App Testing to work to their deep levels of specialism and security.

Second, we’re secure; as far as we’re aware, the only crowdtesting provider to offer the ISO 27001 security certification.  We undertook this higher security standard and certification in order to meet the security standards of our biggest and most stringent clients. It means that you know that your data and your product road map are secure in our hands. 

Third, we’re integrated. We empower our clients to work quickly and easily out of a priority software dashboard which makes sending tests a breeze. But we’ve also worked hard to make that dashboard integration-first – so that you can launch and receive tests out of Jira, Smartbear Zephyr, TestRail, GitHub and even Slack – for the test management and messaging system you rely on. 

Other ways of fulfilling exploratory testing 

There are a number of ways that a business can fulfill exploratory testing: 

Independent Testing Companies: These are specialized testing firms that provide a wide range of testing services, including exploratory testing, to clients across various industries. They have dedicated testing teams with expertise in exploratory testing techniques. We tend to find that they lack the 24/7 availability of crowdtesting and they don’t have as much access to 

Quality Assurance (QA) Consulting Firms: QA consulting firms offer comprehensive testing and quality assurance services, including exploratory testing. They often provide guidance on testing strategies, test process improvement, and overall quality management.

Managed Service Providers: These providers offer end-to-end testing solutions, including exploratory testing, as part of their managed testing services. They typically have a team of experienced testers who work closely with clients to ensure comprehensive testing coverage.

Software Development Companies: Many software development companies also offer exploratory testing services as part of their testing and quality assurance offerings. They combine their development expertise with testing capabilities to provide a holistic approach to software quality.

Independent Software Testers: Independent testers, often freelancers or individual professionals, offer their services for exploratory testing engagements. They can be hired on a project basis or for specific testing requirements.

In-House Testing Teams: Some organizations have dedicated in-house testing teams that perform exploratory testing as part of their overall testing strategy. These teams are typically responsible for ensuring the quality of software developed within the organization.

These are just a few examples, and the landscape of providers offering exploratory testing services can vary. The choice of provider depends on factors such as project requirements, budget, expertise needed, and the level of control and involvement desired by the client.

 

Ready to start?

You can get started exploratory testing with Global App Testing below.

 Contact us to learn more