Continuous Integration: How to Optimise Your QA

continuous integration QA

What does the future hold for QA?

Regular readers of our blog will know that we feel the ability to test at the speed while improving quality is the future of QA.

Read on to learn about:

  • How to blend and optimise your QA
  • How to use QA for growth
  • What it means for Continuous Integration
  • Can QA provide what’s needed for CI?

In the world of ever increasing product releases and development, how do you ensure that your QA is adjusted properly? Can your testing program really keep up with continuous integration or/and agile approaches to development?

Unfortunately, for most organisations and approaches to QA from the past, the answer is no. It doesn’t mean that exploratory testing is dead and it also doesn’t mean that automation or machine learning is the only answer.

What it does mean is that you need to blend and optimise your current strategies so that you can grow your business and testing appropriately.

By blending, optimising and growing your testing powers in the right way, you can count on QA to deliver what the business needs.

QA, as time goes on, should eventually evolve into a business unit that advises the company on how to grow and dominate individual markets. Because ultimately, you need to do one thing – deliver a solid mobile or web application that enables your company to scale to unimaginable heights.

Testing at the speed of release means that you need to automate what you can and then speed up what you can’t.

How to Blend Your QA

Person testing apps

One of the first things you need to do is blend your current QA system. We have a saying here at Global App Testing that there are those who have tried automating all their testing and those who have not. Those that have not are usually in the camp that automation will solve their software testing problems and provide 100% coverage. Then, there are those who have tried automation and realised it is just not feasible to get 100% and therefore understand the importance of blending their QA properly.

To blend your QA, you need to understand that there needs to be involvement from all facets of the company in QA. From the very beginning to the very end of product development, you need to be integrating QA into the process.

Automate what you can after you’ve discovered what needs to be automated.

Functional testing, for example, is a poor candidate for automated software based testing because you can’t program tests for what you don’t know. However, it can and should be blended with automated test case creation based on functional exploratory bugs.

So, for instance, once you’ve completed functional exploratory testing you should then begin the process of automating the testing of those found bugs.

Once you’ve taken the time to blend these kinds of operations, you will quickly get to a point where you can conduct your functional exploratory testing as features are developed.

This enables you then to turn QA into the gatekeepers for release. As we’ll talk about a little bit later, there are many examples of companies that use QA as a release gate.

Developers are free to build and release to CI servers where testing is then conducted by the QA department before release.

How to Optimise Your QA

To optimise your QA you need to begin to look holistically at the role QA plays in your organisation. Beyond the fundamental aspects of software testing, you’ll begin to find areas that QA can deliver additional value to the organisation.

The way you optimise your QA is by improving the feedback loops found within the development cycles at your company.

Improving the speed of the feedback loops will encourage a faster product cycle. Faster product cycles of course mean that you can outpace your competition and win the market. For most companies the product cycle looks generally the same:

  • Idea
  • Requirements
  • Design
  • Code
  • Feature
  • Release
  • Production

So, at each of the stages of the product cycle you should be using QA to deliver feedback at an ever-improving rate.

You can improve the rate of feedback as you move down the product cycle because you’ve built the right automation systems where necessary and the team has been involved right from the ideation stage.

Most people will recognise this as a “shift-left” approach to testing and QA – getting involved as early as possible. “Shift-left” however is only part of the answer to ensuring that you are aligning your QA with continuous integration and development.

Each of the people involved in the product development cycle have their own feedback loops which contribute to continuous integration. The developers, for instance, take product requirements and develop the code for it.

Once this has been done (and during the process), testing is conducted for which feedback (bugs) is delivered. The quicker that this loop can be completed the faster the cycle becomes.

As we narrow down to the QA cycle we apply the same logic. If we can speed up the feedback loops of testing and feedback we can improve the delivery of the result thus resulting in fully optimised cycles across the business.

Using QA for Growth

The idea that QA could be used to grow your organisation a few years ago would have been ridiculed as ludicrous. After all, isn’t the point of QA merely to test software and find the bugs?

As we’ve been exploring already, QA is morphing into much more than just a team of people that looks for problems in software. Some of the smartest companies in the world that we’ve worked with actively use QA as their means for growing their customer base. 

Shifting the lens in how you view QA away from a testing slab to a growth mind-set is the first order of duty. When you shift this mind-set, a few things can happen.

The first thing you will notice is that you begin to see QA as a means to improve your products and services from any perspective in the company. Gone are the days of merely testing software features or code to be replaced with ways to improve the business which results in greater growth.

Consider a well-known social media app and how they utilise their QA functions to deliver relevant and timely feedback. This social media app regularly taps their testing partner (that would be us!) to get regional and hyper-local feedback.

What could be better than determining how your application or website operates in the south of Mexico or Concepcion, Chile? More than just determining how it operates, if you operationalise your QA team properly, you can get feedback on the usability, features and product enhancements specific to that region or area you want to grow in. It’s amazing what can happen when you focus your QA team on growth.

A growth mind-set for QA is a pretty scare jump for most organisations and this is why the growing phase comes after you have already blended and optimised.

Because, if you haven’t worked through the other steps you’re going to end up getting lacklustre results or even worse a convoluted idea of what your QA should be doing.


Owais Peer

Bringing it back to Continuous Integration Testing

Now that we have a firm understanding of how you should approach the testing in your organisation, how does that fit in with continuous integration? The very first step, now that you’ve understood blending, optimising and growing, is to prioritise within that framework. Look for the big wins that make the most impact. In other words, tackle the low hanging fruit first.

Make a list, or identify using the tools that you have in-house, of the top browsers and devices that your customers are using to access your services or products. Then, start to prioritise how you can blend, optimise and grow those areas of the business.

Take for instance the knowledge that 95% of your user base runs on Apple devices in the US, you’ve captured almost 85% of the market and you have an excellent NPS score. In that case, you might not be able to grow much more in the US market from a user acquisition perspective.

You’re going to need to devise a strategy to grow beyond that market and leveraging your QA resources to tell you where should be your approach. According to The State of the Software Testing Profession Report 2016-2017, 66% of respondents stated they were planning on implementing continuous integration and 64% (respondents could have multiple responses) reported that they were implementing continuous testing.

There is a clear shift (pun intended) from both the ground level of testing all the way through to the product management side towards faster overall production and delivery. Couple this with the need for strategy growth approaches globally and you can see how QA provides the knowledge base and reach to enable the shift.

Can QA Provide What’s Needed for Continuous Integration?

We’ve established the importance of QA in not only blending, growing and optimising themselves but also the important role they play in the greater organisation.

Referencing again The State of the Software Testing Profession Report 2016-2017, 32% of respondents reported that they expected to be performing more work outside of testing but within an agile environment.

This work could be centred around learning new skills, working with the product team on feature enhancements or more general business functions. Most importantly, however, the work that QA does over the next few years should clearly be aligned to continuous integration.

This might include finding ways to automate the provisioning of test appliances and unit level tests for example. Furthermore, QA should also be working on automating more complex UI and functional testing within their DevOps environment.

One way that you can automate manual testing such as functional exploratory testing could be using crowdsourced testing platforms for instance.

Crowdsourced testers are available to respond to the ever-increasing demands in a continuous integration environment and could feasibly respond to builds being released to the server.

While automated regression and unit testing would still take place, you can ensure that the edge cases and usability issues are still worked in an expedient manner.

Should QA control the deployments?

One interesting example recently came from a company Global App Testing was working with who advised us that the QA team controlled the public deployments. Developers were free to continue building, developing and coding feature sets and releasing them to the continuous integration servers.

Once they hit the continuous integration server, the QA team took over when and where they got released. 

What this did was fully empowered the QA team to make decisions based on the quality and functionality of their application and website whilst freeing the development team to focus on what they do best.

It was also an interesting approach because if a feature was not performing well after multiple releases, the QA team retained an almost ultimate veto power on that feature. If they had too many bugs for example, the QA team could push that to a further review.

Now, this approach might not be appropriate for all teams and organisations but it does highlight just how innovative you can be with the application of QA to your continuous integration approach. Should you try this approach?

Ideally, you need to make sure that if you attempt something like this your entire company is committed to its success as a methodology for how you develop. If even one iota of your company is misaligned it could cause havoc.

For those companies who are making a gradual switch to continuous integration, you would best be served by slowly moving towards this depending on how mature your DevOps environment is.

You could certainly trial something like this on a per feature or per team basis to determine effectiveness. As always, focus on testing and iterating these kinds of approaches until you find the right fit for your organisation.

Software Testing


We’ve covered quite a lot of ground in this article. The combination of continuous integration and QA is complex and there is no one-size fits all approach to how you go about amalgamating the two.

What you do need to do and be aware of is that your testing process shouldn’t be slowing you down. If you want to learn more about how crowdsourced testing can help you, download out free Crowdsourced Testing White Paper.


Was this article useful?

Great! We'd love to send you more articles like this



Join 24,389 people who already get the latest QA advice - unsubscribe any time!