When you think of the word automation what springs to mind?
For many, the word automation conjures up images of tasks being completed at lightning-fast speed. But automation, as we know, isn’t that simple. It requires a budget, planning, set up, and maintenance. This can leave many companies hesitating to automate parts of their business, wondering whether it may take more time and effort than simply completing tasks manually.
Still, automation, in some capacity, is what many tech-first companies strive for.
In QA, automation testing can mean faster test results and a greater volume of tests. It can also reduce the margin of human error and mean that you can decide to run a test at any hour of the day, even when you aren’t in the office. This is music to the ears of any CTO looking to streamline their process.
So, what is the best way to start automation testing?
What can potentially hold you back, and how do you make the first steps into integrating automation into your strategy?
At Global App Testing we’ve worked with companies of all sizes to streamline and improve their QA strategies. Here’s what we’ve learned about automation:
Why do we hesitate to automate?
As mentioned at the beginning of this article, there are a number of reasons why you might hesitate to make the move towards automation testing:
Automation tools can be expensive, especially for a smaller startup without a huge budget. Some organisations simply lack the resources to buy new software and hire more staff to manage it.
Every CTO knows that automation isn’t as simple as downloading software and watching it go. You need to have the time within your team to write test cases and set them up. Automation tools also require maintenance and rewriting of test cases when updates happen. This can lead to a lot of time spent writing code, and if your team is strapped for time, this could create more stress than it solves.
A study conducted by IBM about the cost of manual testing vs. automation found that there were three main situations where automation was more efficient than manual testing:
- The automated test case is expected to have a relatively long life without needing to be changed or edited.
- The test case is comparatively easy to automate, meaning that it can be created from a generalized manual process; the more complex the task, the more difficult it is to automate.
- The comparative cost of automating is lower than that of executing the test manually.
Not every testing task, therefore, is suited to automation, and deciding on what needs automating is a key step in the process. In some cases, it may not make sense at all to automate your testing at this current moment in time.
If your product is relatively new, or in the ‘validation’ stage, your team's main focus is to deliver an MVP and find a product-market fit. Spending time and budget writing automation tests for a product that could change in a month's time just isn’t efficient at this stage. Automation, therefore, shouldn’t be discounted completely but put on the back burner for when your product is more secure in its foundations.
Discover more about testing according to product maturity here.
Size of QA team
If your team is relatively small and preoccupied with a large amount of testing, you could be hesitant to implement a QA strategy that the team doesn’t have the capacity to undertake. Not every company has the budget to expand with new hires, so your team size can become an issue. CTO’s often report that resource allocation is a top concern for them and QA is no exception (link).
While these hesitations are often valid, there are ways you can troubleshoot the blockers and start your automation process.
How to start with automation testing
Decide what needs automating
Not every part of your testing structure needs to be automated. So, it’s important to take some time to establish where in your release cycle automation is going to be the best option.
Sit down with your QA team and go through each part of your testing process. Where do tests seem repetitive? Where does your team feel they could speed up the process?
Ask the following questions:
How laborious is the test?
If a specific test requires a huge amount of manual data input, it could be a very safe bet for automation. Adding endless data entries into a manual test is extremely inefficient, so automation is likely a more time-efficient option.
How accurate are your testing results?
If you are constantly having to cross-reference your test results to check for inconsistencies, human error could be interfering with your testing process. Automation can reduce the likelihood of this happening and run hundreds of test cases to discover bugs that may be missed by the human eye.
How often do you repeat the test?
If you have been repeating the same test time and time again, automation could save you a lot of tedious manual work. The time spent writing a test case will likely be shorter than having to conduct manually repetitive and tedious testing. This will open up more time in your team’s schedule to start new projects or work on new features.
Does the test have longevity?
If your test is likely to be the same in 6 months time, and what you are testing isn’t expected to change, automation will be a time and cost-saving exercise. Locate the tests in your strategy that are rigid and unchanging, and set about automating them.
Do you require validation of a predetermined idea?
If you know what you want to test exactly, and points 1- 4 apply, automation testing would be a fantastic option. Automation either confirms or denies pre-written test cases, and pre-determined ideas, so if you know what it is exactly you are wanting to test automation testing is fantastic for reducing the margin of human error and delivering high-quality results.
Decide what doesn’t need automating.
A well rounded QA strategy uses a blend of manual and automated testing.
Fine-tuning your testing means deciding which tests are better suited to which different testing strategies. If your testing tool kit is wide-reaching, you will get a wider testing coverage, and catch more bugs.
With that in mind, deciding what doesn’t need automating is a key step in starting to incorporate automaton testing. Not every method of testing can be automated: some require human creativity to be conducted successfully.
The following methods cannot be automated:
Exploratory tests are tests that explore an app to try and discover potential bugs. These must be conducted manually by human testers, as they require creative innovation to decide what part of the app wants to be tested. Testers will not follow a pre-determined path but have the freedom to decide how they navigate the app.
UX/ UI tests
User Experience testing is all about testing different parts of the user experience. This is best conducted by manual testers, as you want to test how a real-life user would interact with your product. Any user experience issues will be picked up by a tester simply experiencing what is it like to navigate your app.
User Interface testing, similarly, is about testing things like design elements, and typography. These require testing by the human eye.
Start with the automation process
Once you have decided which parts of your testing process will be automated, and remain manual, you will have the basis for beginning to implement your automation testing strategy.
We spoke to tech industry professionals about how they start the automation process, to provide you with some top tips about getting started.
“The first thing that I do when introducing automated testing to a code suite is to ensure that there is Continuous Integration setup that supports the automated running of tests. A good Continuous Integration process will ensure that all tests in the code suite are run and passed before a branch or feature is merged upstream. This ensures that the efforts that you put into building tests are not futile and that the test suite is fully maintained. Without this sort of infrastructure, it is easy for a developer to merge upstream, and eventually into master, with failing tests. This would defeat the purpose of having a test suite in the first place.”
- Michael Frederick, CEO of Flatirons Development
“Technology has brought customers’ loyalty under constant threat. How do you turn this threat into a business opportunity? Answer: Prime your software delivery process for continuous improvement. Step away from traditional, mostly manual, testing or solely look at automating the regression cycle.
When organizations implement Continuous Testing, the focus needs to be on putting the systems, processes, and automation in place that will make the most impact. The majority of organizations still automate less than 25 percent of their testing and the lack of automation is considered as one of the major roadblocks in evolving towards Continuous Testing. If CT is to succeed, higher automation levels are critical within every activity. This means from test data management to environment provisioning and result feedback analysis, an approach that incorporates automation needs to occur. By seriously striving for the appropriate automation levels and further augmenting automation efforts through the use of AI-powered bots already in the market today, it becomes possible to optimize, automate and accelerate the entire test cycle.”
- Manish Mistry, CTO of Infostretch
“Creating automated testing starts by first winning over the developers and ensuring there is time for it. Developers always feel the pressure to create more features as fast as possible, and often have little time to test. Allowing them sufficient time to create some basic automated tests (about 20-30% of development time) is mandatory to not only to create automated tests but doing so empowers the developers or the Quality Assurance (QA) team to further add or refine tests.
By having the developers create the basis of automated testing for their features and software, it reduces the friction to, later on, add new tests. If you have a QA team then you’ll want to make sure they have established meetings with the developers every sprint to communicate about the feature and current tests. This practice will ensure you promote testing from the beginning and ease over potential conflict between developers and the QA team.”
- Colin Ma, Founder, Digital Software Products
Need help with your QA strategy? Book a call with one of our Quality Consultants for a free consultation.