Every CTO or VP of engineering is constantly looking for ways to boost their current QA strategy and deliver quality at speed, time and time again.
Especially in a tech world where 71% of app users churn within 90 days, engineering teams are under a huge amount of pressure to deliver great quality products that impress hard-to-please customers. And with lightning-quick SDLCs now being added into the mix, it seems that any way to streamline QA processes is music to the ears of tech professionals.
That’s where automation comes in.
Automation is the buzzword on everyone’s lips in the world of software development and beyond. The mantra seems to be: if you can automate something… automate it.
Repetitive tasks are now being carried out by AI rather than humans, and so much of our daily lives can be automated, right down to how we make our coffee in the morning. Those who champion automation hail the speed, ease and scale at which it can be carried out.
So, should you be automating your testing too? Let’s find out!
The case for automation
There is a clear case for automation: you can conduct pre-scripted tests at lightning-quick speed, receiving fast results. That means your SDLC can keep moving at a rapid pace, with automated testing tools fitting seamlessly into your existing workflows. It’s fast, and it can also save money too. The money spent on hiring manual testers to carry out repetitive testing can be saved, replacing the process with a reusable automated test script.
But don’t just take our word for it. Hear from the experts:
“The two primary benefits of automated QA testing are fast feedback on new code and increased bandwidth for testers to investigate more complex potential issues. Developing unit tests to QA new code takes time. However, once these unit tests are complete, they unlock rapid feedback on new code. This ensures the team can fix any issues before switching to a new project. Additionally, automated QA testing allows testers to focus on more complex investigations. The QA team can also use its newfound bandwidth to build new features into the testing platform.”
- Bruce Hogan, CEO of SoftwarePundit
“Automated testing lets us develop better digital products with less time and money. We get broader and more thorough test coverage that's done in less time than manual QA. It may seem like a significant business expense to implement but the cost savings compound over time. Instead of rigorous manual QA, tests run automatically to give us specific QA feedback.”
- Husam Machlovi, Managing Partner, With Pulp
“Automated QA testing is really the only way to scale a quality technology product. As products mature, by definition they become more complex. There are more features, there are new team members, old team members leave, and as always, people are only human and make mistakes. The only way to make sure you’re not constantly fighting regressions and don’t have to hire a massively outsized QA team doing slow manual QA labor, is through automated QA. The workload is just impossible otherwise. Automated QA also gives you consistency in testing, which directly equates to stability.”
- Jorge Perdomo, Co-founder of goTenna
As outlined by our experts, automation has clear benefits. It can save time and money, giving you lightning quick results and enabling you to run hundreds of tests at once. It can also help you scale, providing larger scale testing without the associated recruitment headache.
So, what now? The key to success is in the planning.
What tests should you automate?
Let’s be clear. Automating every single part of your QA process isn’t going to solve every problem and deliver a bug-free app that customers can’t get enough of. Unfortunately, although it has its benefits, automation is not a silver bullet. We wish it was!
Making automation testing work requires a rigorous assessment of your current QA process. It’s necessary to break down each part of testing that you do and decide whether it should be automated or not.
You need to pinpoint the tests that are going to benefit the most from being automated. Once you have outlined your plan, then you can start implementing automation.
So, how can this be done?
According to a study conducted by IBM, there are three key scenarios where automation is more preferable to manual testing:
- The automated test case is expected to have a relatively long life without needing to be changed or edited.
- The test case is comparatively easy to automate, meaning that it can be created from a generalised manual process; the more complex the task, the more difficult it is to automate.
- The comparative cost of automating is lower than that of executing the test manually.
When looking at your current QA process, decide whether any of your tests fit into one or more of these categories. If they do, investing in automation could save you time or money down the line. After all, wasting the precious time of your team conducting expensive manual testing that could be automated is not something any leader wants to do.
Now you’ve decided what you’d like to automate you’ve taken the first step to streamlining your QA. The second step is deciding what, in fact, should not be automated at all.
What tests should you not automate?
The short answer is… creativity!
Some forms of testing simply require human creativity to work. In exploratory testing, for example, testers explore an app to uncover bugs, behaving as a user would do in order to test the quality of your product. This is the investigating side of testing, and (right now) it is impossible to automate that human creativity and insight. But who knows where we’ll be in a few years…
In fact, there are a number of testing methods that cannot be automated:
- Exploratory tests
- API tests
- UI tests
- UX tests
That’s because these testing methods are subjective. UX tests, for example, are not about yes or no answers, and UX testers don’t follow a strict path. The tests are about the subjective experience users have.
This also depends on your product maturity
Another factor to consider is the maturity of your product. In Leading Quality, Ronald Cummings-John speaks about how your testing strategy should adapt as your product changes and grows.
Testing according to your product maturity is all about making your QA work smarter, and ensuring you don’t waste precious time and resources. For example, if your product is at an early iteration with a very small user base, automated testing should not be your primary concern. That’s because you are busy creating a minimal viable product, and ensuring your app or website will succeed in your chosen market.
Automation at this point would mean writing out detailed test cases that could be irrelevant if your product changes a month later. The expense and time spent on automation tools would feel like a foolish mistake.
However, as your product scales, your testing should scale with it. As your user base and your product grows, ensuring you continue to deliver quality is of the utmost importance. Automation at this point could be a really smart decision, as you can speed up your testing process.
Do you need automated QA testing?
In summary, deciding whether to use automated QA testing is all about analysing your current process, and seeing where automation fits.
Ask yourself the four following questions when deciding whether you need automated QA testing:
- Do you expect the tests to have a relatively long life without needing to be changed or edited?
- Would it be cheaper to automate a test than conducting it manually?
- Is the test comparatively easy to automate?
- How mature is your product? Do you expect a lot of features to change in the coming months?
These pointers will hopefully guide your decision making and inform you whether automated QA testing is right for you.
Automated testing should be just one powerful tool in your testing toolkit. Just because the method works for one test, does not mean it should be the only method used.
In fact, from working with tech teams from the likes of HelloFresh, Etsy, Google and more, we have found that a blend of manual and automated testing is the best way to achieve full test coverage and deliver the highest quality product.
At Global App Testing, we use a blend of crowdtesting and automation to increase the testing capacity of our users and find the critical bugs impacting their customers.
Book a free consultation with one of our growth specialists today to discover more about what we do.