agile testing guide image v4.5.png

Introduction

Agile testing has become a critical part of application lifecycles and has had a significant impact on software development, testing and quality assurance. It has also gained widespread acceptance as a crucial driver for the delivery of high-quality products.  In this article, we take a deep dive into the world of Agile testing to better understand how it works and how it can help you.

 

For the first decade of its existence, Facebook's development mantra was infamously "move fast and break things." It was the essence of the glamorized "just ship it" culture endemic in Silicon Valley. But in 2014, Facebook CEO Mark Zuckerberg toned down the hasty rhetoric with a new company slogan: "Move fast with stable infra." In his inimitable syntax, Zuckerberg explained why the change was necessary, saying, "What we realized over time is that [the old mantra] wasn't helping us to move faster because we had to slow down to fix these bugs and it wasn't improving our speed."Developers often have a single goal in mind: to create functional code that meets the client's specifications. In short, they want to move as fast as possible while breaking as few things as necessary.In a perfect world, this would work well. We do not live in a perfect world. Customers are not always good at creating the specifications for a job, meaning that the end product might not meet their expectations. Or a vital function of the product might be one of the few things that breaks, resulting in unhappy customers.

Agile Testing in numbers

89%

Of development models used are Agile or similar

1 in 2

Believe Agile skills are very important to be a good tester

40%

Of testers use CI & CD in All projects

 

The shortcomings of traditional testing

To keep things from breaking in the customers’ hands, testers attempt to break it first - and then have it fixed.

In the traditional waterfall method of development, the sequence of events is Requirements > System Design > Implementation > Integration and Testing > Deployment of System > Maintenance.

With this method, the next step does not begin until the previous step has been fully completed, which means testers do not receive the product until late in the development cycle.

Bugs caught so late in the process are difficult and costly to eradicate from the product.

Testers who enter the process at this juncture are also unable to ask the right questions and perform the right tests, because there is little feedback from other members of the development team (who are sometimes viewed as ‘the enemy’) or from customers.

Testers are forced to simply wait for the product to come down the assembly line, then use a narrow set of skills to decide whether it should be kicked back to a previous step in the development process.

❝ Broken things are fixed, but movement is not fast.❞

Agile testing delivers

By contrast, Agile testing operates under the philosophy that testing is a bona fide part of development, on a par with coding, and the singular focus is to deliver the best product possible to the customer.

Testing is integrated directly into the development process so that bugs are discovered as early and as often as possible. As a result, testers can identify problems at every point in the development process, moving the product quickly towards release.

 

Principles of Agile testing

In the book, Agile Testing: A Practical Guide for Testers and Agile Teams, Lisa Crispin and Janet Gregory distilled Agile testing into 10 principlesSince being published, these 10 principles have been widely accepted as the foundation for Agile testing;

  1. Provide continuous feedback. Agile testers do not simply test constantly. They are also responsible for distributing the results of those tests and facilitating the provision of feedback from customers to developers in order to create a more robust product.
  2. Deliver value to the customer. While this is the second principle, it is the paramount principle. The end goal of every action taken by an Agile tester is to create the best product possible for the customer.
  3. Enable face-to-face communication. The Agile tester's role is to reduce confusion and errors by communicating directly with developers as well as enabling customers to communicate directly with developers.
  4. Have courage. Developers can be defensive about their work. To meet their goal of providing value to the customer, Agile testers must fight for the changes and fixes that need to be made.
  5. Keep it simple. Agile testers act on the aphorism that simplicity is the ultimate sophistication. For testing, that means performing only those tests that are necessary, and all tests that are necessary. For the product, that means delivering the simplest possible product that delivers the most possible value.
  6. Practice continuous improvement. Agile testers are keen learners; they are never done learning how to do their job better.
  7. Respond to change. Agile testers are adaptable and flexible, keeping up with changes in the product and the marketplace.
  8. Self-organize. Instead of waiting at an assigned seat on the assembly line, Agile testers spring into action at every point in the process. They actively seek out problems and bring people together to solve them.
  9. Focus on people. Agile testers are collaborative, preferring human interaction to technology. Their focus on people enables them to deliver a product that both has utility and is usable.
  10. Enjoy. No one is as successful at meeting their goal as when they enjoy what they are doing. Agile testers who enjoy the work are able to deliver the greatest possible value to the customer.

 

Profile of an Agile tester

coder or developer at desk

From those principles, a profile of the ideal Agile tester can be derived. In general, Agile testers possess the following characteristics:

  • Acute communication skills
  • A diverse, technical-based skillset
  • An ability to effortlessly collaborate with others
  • A willingness to embrace change
  • An ability to liaise with both technical and business people
  • Broad experience in exploratory testing
  • A results-oriented personality
  • A passion for delivering value in business

 

Agile testing methodologies

A myriad of methodologies have been developed for Agile testing. Below are four of the most popular methodologies currently in use. While no single methodology is perfect for a specific product, these frameworks are useful as starting points from which to generate a bespoke approach:

 

Acceptance test-driven development

ATDD embraces the collaborative nature of Agile testing, bringing together customers, developers and testers to create acceptance tests from the customer's point of view. Only once these tests have been created is the corresponding functionality developed. This gives developers direct insight into what customers want and how the product will be used, removing ambiguity from the process and reducing the chances of large errors being made.

 

Behavior-driven development

BDD is based on and enhances test-driven development and acceptance test-driven development. Using their structure adds the identification of correct business outcomes and performs tests based on those preferred outcomes.

BDD has five steps:

  1. Describe the behavior
  2. Write the step definition
  3. Run and fail
  4. Write code to make the step pass
  5. Run and pass

 

Exploratory Testing

Exploratory testing is a cyclical method, progressing from test design > test execution > analysis > learning before beginning the loop again. The tests themselves are not scripted; instead, they are generated by Agile testers as the product is explored, requiring the tester to make full use of his or her unique skill sets.

Exploratory testing is the closest testers get to interacting with a product precisely how it will appear ‘in the wild,’ and it allows testers to identify bugs that would not be found through other testing methodologies.

 

Session-Based Testing

Like BDD does for ATDD, session-based testing builds on and refines exploratory testing.

The strength of exploratory testing - the creativity of the people who do it - can also be its greatest weakness. Session-based testing attempts to remedy this by adding structure. First, before a test session is begun, a charter is created. Second, uninterrupted testing sessions take place, focusing mainly on a single charter. The entire session is then reported on, and the manager is debriefed after the test. The additional structure ensures that all areas of the product are thoroughly tested.

 

Agile Testing Quadrant

With these and other testing methodologies, it can be difficult to assess which type of test should be run, how often it should be run, when it should be run, and who it should be run by. Gregory and Crispin created the concept of Agile testing quadrants, which provide a taxonomy for tests. According to Crispin, the two left-hand quadrants help teams know which code to write and determine when they are done writing it. The two right-hand quadrants help teams learn more about the code they have written, providing feedback to the left-hand quadrants.

 Agile-Testing-Quadrants-GAT-cropped


Q1 - The Automated quadrant contains tests that are designed to improve the code of the product being created; they are performed to help the team create a better product. 

Q2 - The Automated & Manual quadrant contains tests that are designed to improve the business outcomes of the product being created; they are performed to help the team create a product that drives value for the business and for customers. 

Q3 - The Manual quadrant contains tests with the purpose of providing feedback for tests in quadrants 1 and 2 by testing the product and user experience to ensure business outcomes. 

Q4 - The Tools quadrant contains tests that use technology to ensure the code fulfills all nonfunctional requirements such as security and compatibility.

 

Benefits

There are three simple benefits to adopting Agile testing: a happier team, a higher-quality product and faster delivery. But that trifecta is worth the effort put into developing an effective Agile testing framework.

 

A higher-quality product...

Agile enables testers to detect more defects earlier in the development process.

One of the Agile principles is ‘continuous feedback.’ The doctrine of starting testing concurrently with development means that bugs can be eliminated soon after they are created. Testing also involves every member of the development team, so the skills of both developers and testers are leveraged in the pursuit of a perfect product.

Another outcome of continuous feedback combined with early and frequent testing is testers developing an intricate knowledge of the product. Depending on the methodology of testing used, they can combine that knowledge with customer input to help developers create a superior product.

 

Delivered faster...

With waterfall testing, the initial stages of development and eventual release onto the market are separated by months, if not years. As a result, features or the entire product can be completely irrelevant by the time it reaches customers.

Agile testing methodology both compresses the development cycle and constantly provides customer feedback, ensuring the product adapts to the market during development and reaches customers as soon as possible.

 

From a happier team.

The last principle on the Agile testing list is no mistake: enjoyment. Agile testing necessitates close interaction between all members of a team, creating a happier, more enjoyable, and more productive workplace. Developers, testers and customers work side by side to create the best product and the most value possible.

Crispin and Gregory say it best: "A team that guides itself with Agile values and principles will have higher team morale and better velocity than a poorly functioning team of talented individuals."

 

Possible pitfalls

However, no system is perfect. Improperly implemented, Agile testing can weaken team structure and product development, preventing a viable product from ever being released. Even when properly used, all Agile methodologies have their weaknesses. For example, exploratory testing can lack the structure necessary to ensure a product is comprehensively tested; ATDD accounts for customer feedback, but not for business outcomes.

The emphasis Agile testing places on people can also be its downfall. If Agile testers are excluded from the team that they need to be closely integrated with, they are rendered impotent. If a single skilled Agile tester leaves, it can prove to be a major setback for the development of the product.

Finally, since everyone in the team performs testing, the muddied hierarchy can lead to confusion and conflict.

 

Tips

With dedication, each of these pitfalls can be overcome and the three powerful benefits experienced. The first step towards successful Agile testing is determining when Agile testing should not be used. Blind adoption of Agile testing can result in a weak, crash-prone product.

Here are a few guidelines for cases in which Agile may not be the best way to test:

  1. When the project scope is crystal clear and very unlikely to change
  2. When the project is governed by a single stakeholder with minimal requirements
  3. When the people on your team lack the deep, broad skills necessary to perform Agile testing.
  4. When the customer insists on using a traditional waterfall approach to testing

Once you have determined that Agile testing will benefit your team, your product and your customers, you should spend as much time as necessary to pick the right methodology and to create a process for testing using the four-quadrant model.

To counteract the possibility of testers' exclusion, testers should work in as close physical proximity to the developers as possible. They should meet with them often to see what they are currently working on and to give them a chance to review the tests that have been developed.

Testers can open doors for themselves by providing useful feedback based on interactions with both developers and customers. In short, they should make themselves indispensable to developers in order to be able to perform their job well.

The greatest thing that can be done to ensure the success of Agile testing for a product is to hire people who have the essential characteristics of an Agile tester, and to inculcate a culture of self-organization and independent thinking in the entire organization.

That environment will naturally result in ‘stable infra’ without sacrificing speed, resulting in happier workers delivering a better, more valuable product - faster - to a satisfied customer.

Download our Facebook Case Study & find out how we tested 5,000 Facebook apps in 4 weeks

Schedule a Demo