How to Build a Culture of Developing Quality Products

(Lessons from Airbnb, Sprint and Snapchat)

Quality Starts at the Top


The TLDR version:

  • Educate your company and teams on why they need to invest in quality
  • Collaborate with other departments to develop a shared understanding of quality
  • Create your quality narrative to get organisational buy-in
  • Cultivate quality advocates in your organisation to share best practice

 

In February 2017, Snapchat, already under pressure from an underwhelming IPO, had its back up against the wall after a grilling from investors, namely dealing with the big elephant in the room:

Why was user growth slowing?

Until that point, the Snapchat Android app was buggy: it only added six million new users during that quarter, a third of what had been added during the previous three months, with various complaints coming from users about the number of bugs they had to deal with every time there was an update.

Developers Snap Inc. had decided to prioritise iOS in the early stages of the app’s life, due to greater revenue per user and the dominance of iOS in key target markets.

This indicator shows how management teams can seriously impact quality.

Unfortunately, at the same time, Instagram swooped in to pounce on Snapchat’s mishaps in the Android market and capitalise while it could. Considering Android’s market share stands at 77% with over 2.5bn users globally, it seems obvious that Snapchat’s lack of focus on Android caused long-lasting damage to the brand.

android-marketshare-qaops

If Snapchat had installed a culture of quality across its product line from the beginning, its international market share wouldn’t have suffered as much as it did when others came along to compete.


The responsibility for quality is changing


It’s never been more important to deliver high-quality products that customers love.

Companies that rely on the old ways to ensure quality, are likely to find themselves out of date and out of touch with the new ways companies are working to stay ahead.

Previously, quality was thought of as something “the testers had to do”, but this led to sub-par results. No one had direct ownership over quality, which led to increasing tension between different departments.

Without focusing on your whole company’s approach to quality, you increase the risk of releasing a buggy product and subsequently putting your brand’s reputation on the line - this is the first pillar of QAOps.

 

From ‘quality standards’ to a focus on customer experience


As the speed of development quickened, teams picked apart their current processes and implemented new workflows. This involved collaborating with multiple departments to speed up the delivery of software to the customer, leading to the software engineering culture that we now know as DevOps.

Each evolution of software development methodologies indicates a trend towards tying quality with the customer experience.

This approach is a far cry from the days of quality being defined as a set of standards (as it was in the 1940s during World War II) to ensure unchecked bombs didn’t explode on the factory floor.

bomb-testing

According to a study by Walker, by 2020, customer experience will overtake price and product as the key brand differentiator. Customers expect more; it’s easier than ever to move between vendors - even if you’re in the B2B software space. The rise of the subscription economy has led to buying power shifting further towards the consumer.

We all know the stories of markets being disrupted by newcomers who prioritise the customer experience, but my favourite is the story of Blockbuster - the video rental market leader that, in 2002, had over 8,000 stores.

Who doesn’t remember paying a late fee for returning a VHS tape after three days?

This exact situation is what fuelled Netflix CEO Reed Hastings to build something better. Netflix is what ultimately sealed Blockbuster’s fate, but its tone-deaf approach to customer experience is what allowed this disruption to succeed.

Developing and executing a quality strategy must become everyone’s responsibility. Gone are the days when teams could sit in their respective silos with limited interactions with other departments.

The highest-performing teams take quality seriously as an organisation and have “quality owners” or “quality advocates” who evangelise the benefits of the organisation taking responsibility for its work and what it delivers to the end user.


What are the benefits?

quality-narrative-3-benefits

  • Better team alignment: When you decide to have a quality-focused culture that involves everyone, every department has to have clear expectations of what quality means to them. This requires conversations with different departments to understand their baseline - these discussions on quality can lead to them taking a more collaborative approach to bake quality into the product from the beginning and iron out potential issues beforehand, while aligning teams around a common goal.
  • Better quality products: When every individual internalises the responsibility to deliver quality, as opposed to quality being something that’s dealt with at the end of the development process, this leads to the team developing higher quality products as everyone has taken ownership at each stage of the process.
  • World-class customer experience: When QA is everyone’s responsibility, it becomes obvious for the team to prioritise the customer experience over everything else. If a team adopts a quality-focused culture, it is more likely to succeed in delivering the best possible customer experience.

How Sprint is driving quality forward


What does embracing quality look like in reality?

One company that comes to mind is none other than Sprint, the fourth-largest mobile network operator in the United States, which serves over 55 million customers. The company was able to take a top-down approach to developing a culture of quality.

Sprint realised that it wasn’t delivering products to the standard it knew it had to in order to compete. It had to completely transform how it tested in order to get the whole organisation pulling in the same direction about how quality should be - the Director of Release, Aaron Haehn, was tasked with driving this forward.

There was a fundamental disconnect between what executives expected and what the testing team felt was necessary. Aaron had to figure out what the organisation needed while getting other departments to embrace their role in delivering product quality.

Aaron initially worked with his teams to get everyone on the same page based on what quality meant to them, and to increase the understanding across the organisation of why quality was important.

Once that was defined, he had to get other departments on board with his vision of bridging the gap between what testers talked about and what the management team cared about.

This wasn’t easy - as you’d expect, other departments have their own thoughts about what quality means to them, but through conversation and educating others as to the role they played to deliver quality, he was able to change mindsets.

Aaron recognised that he wouldn’t have been able to do this without executive backing - luckily for him, he had support from the engineering leadership to improve quality for their applications. Had the executive team not seen the value of investing in a quality strategy or recognised its responsibility to champion quality, Aaron and his teams wouldn’t have been able to achieve the results that they did or get the required support from the whole organisation.


Airbnb’s “bottom-up” approach to embracing quality


However, taking a top-down approach isn’t the only way to embrace quality.

In any case, such an approach may be out of your control and too complex to implement due to other company priorities.

Airbnb took an alternative, ‘bottom-up’ approach, with a group of committed individuals deciding to lead by example and take matters into their own hands.

At the beginning, code was hitting the production servers without many checks and this was becoming problematic as the team scaled.

After a while, a few engineers decided to start submitting pull requests (PRs) for their changes. This required engineers to review other engineers’ code before it hit production, to ensure no obvious bugs would be released to production. It wasn’t something they were forced or incentivised to do; they just decided to make a change in the quality of the code that was being pushed out.

It quickly became clear to the team that this process of peer review led to less bad code hitting production, and therefore fewer outages. Additionally, all new hires were briefed on best practices that involved submitting PRs, which over time, led to everyone else adopting the same practices.

Shortly after, the engineers decided to include tests with any new code, while also beginning to educate teams about the importance of testing: they spoke at meetups, held office hours and shared recommended readings to increase their teams’ knowledge. New hires were also made into champions of testing, while further investment was made into the testing infrastructure, making it easier to write and run tests.

By taking this grassroots approach, the Airbnb team built good testing habits and installed a culture of quality across its teams.

These initiatives helped the company to release high-quality code, installing confidence prior to each release while also impacting business growth positively.


How to create a culture of developing quality products

How do you go about creating a culture of quality within your company?

From working with some of the largest and fastest-growing technology companies to help them to create their quality strategy (and subsequently develop a framework that we call QAOps), we’ve noticed a number of patterns in the ways they think and operate (these will all be revealed in subsequent articles).

There are three major areas teams need to invest in:


  1. A quality strategy that involves every department
  2. A quality narrative that helps you to get organisational buy-in
  3. A team of quality advocates to continually champion quality
quality-strategy-narrative

Quality strategy - assess the current landscape and where you want to be


A quality strategy doesn’t just focus on what you will do to ensure quality - it also acts as a roadmap for where you are right now, where you want to go and who’s going to help you to do it.

The first thing to do is assess the current landscape - what does quality look like to you and your team?

This will help to inform your strategy and what’s needed based on your starting point. For example, you may have some consensus with a small group of developers who believe that the QA team shouldn’t be the gatekeepers of quality. Or, if you’re the only person to believe quality is important, you need more buy-in before implementing any initiative.

When we did this exercise with an organisation recently, we asked them what they weren’t happy with and were trying to move away from. This really helped them to give an honest perspective on their current situation, which in turn, helped to inform their strategy for what they wanted to achieve.  

Once this is done, you can start sketching where you want to be:

  • Where do you want the organisation to be in 12 months’ time?
  • How do you want quality to be perceived?
  • How do you want your organisation to be seen in the wider market?

 

This is especially important as it gives you your destination and provides teams with a clear vision of where you want to go and what they’re working to achieve.

When we spoke to the aforementioned organisation, after an intense discussion, they realised no one else truly appreciated their responsibility for the quality of the product. Teams were working separately and there was little collaboration. Additionally, management teams didn’t understand the benefits of investing in quality.

After our discussion, they came up with a group of “vision statements” of how their organisation was to have changed in the next 12 months:


  • Everyone in the company feels responsible for the quality of the product;
  • The management team has a true understanding of the importance and value of a quality infrastructure;
  • We are investing in improving the quality of our products from a people, process and technology perspective.

A clear understanding of where you’re going is the first step to establishing a quality narrative in your organisation - as we will cover in more detail in the next section.


Each department’s role in safeguarding quality


What are some of the activities we’ve seen great teams execute on in order to invest in quality? Here are some of the activities they can do to directly influence quality:


  • QA

    • A QA team should be doing more than just writing and performing automated and exploratory tests. These days, teams are encouraged to ‘shift-left’ by becoming involved in earlier phases of the product life cycle. QA teams can do this by working with the product and design teams to ask questions throughout these first phases to help the team to incorporate quality from the beginning.
  • Development

    • A development team can introduce relevant safeguards to operate in a much more effective and cohesive way. Developers must think about the best architectural patterns, create consistent code guidelines and utilise tools like static code analysis. It’s vital that development teams take ownership of, and accountability for their work.
  • Design

    • The priority for design teams is to ask some fundamental questions: how will this feature be used and will it make sense to the user? Will this feature cause issues with other features and even confuse the user?
    • Asking these questions beforehand and being able to articulate this to other stakeholders will help with building in quality from the beginning.
  • Product

    • At our own company, product managers are constantly asking questions such as: are we building the right thing? Does what we’re building fulfil the need of the customer?
    • By incorporating ideas learned from observing customers and actively listening to their feedback, you can improve product quality. Clear user stories and acceptance criterias before development takes place will encourage this.
  • Customer Support

    • There’s a lot to learn from customer support teams. They are the frontline troops that your customer will interact with if they ever have any issues. The ways customer support teams capture, categorise and relay the issue the customer is having back to the organisation -  especially to the product and development teams - will affect quality.
    • I realised how powerful a support team can become for an organisation when I heard a story about the early days of HubSpot, as relayed by David Cancel, the CEO of Drift. He team wanted to fix the most important problems that customers were dealing with, but needed the help of the support team to organise and prioritise what they needed to focus on. While organisations can give this to product or engineering teams to do, the support team will be able to bring a perspective that’s closer to the customer.
  • Sales & Marketing

    • Digital marketing teams are already doing QA testing, whether they realise it or not.
    • Any time digital marketers test out landing pages or create a new piece of downloadable content and put it out into the world, they go through the entire flow to make sure everything is working as it should - in other words, they are ensuring quality.
    • Integrating a solid quality strategy into your digital marketing strategy will improve the prospect and customer experience and increase the chances of your marketing initiatives being successful.

 

What about cross-functional teams?

Cross-functional teams - defined as team members from different disciplines who are working on a singular goal - can arguably have an even greater impact than individual teams can, to influence quality in the organisation. Here are some of the areas they can focus on:


  • Developing a shared understanding of quality:

    • It’s important to develop a shared understanding of what quality means to each team to encourage transparency and collaboration between teams. To do this, teams must be aligned as to what ‘quality’ means to the organisation. Teams can start by asking questions such as: what is everyone’s perception of quality? How does each department believe others currently think about quality? It’s important that this conversation is brought up in order to avoid misalignment later down the line.

  • Cross-functional pairing sessions:

    • Individual team members from various departments can get together on a regular basis to work on a singular issue. This brings unique perspectives together in order to achieve clarity across teams.
    • Pivotal Labs, a software company, has its developers and designers ‘pair’ together for half a day to fix design tweaks that would’ve been a frustrating experience if they’d just communicated all of the changes via their internal chat system. This saves time and builds empathy between the two departments to know what can be done next time to make the other team’s life much easier.

Your company already has a quality narrative


Going back to the Sprint story, what Aaron did when he got other departments on board was that he understood what their goals were and bridged the gap between what testers talked about and what the management team cared about. In short, what he did was change the quality narrative within the organisation.

Remember what we said about working out where you are right now in order to create a quality strategy?

This is your current quality narrative.

For some companies, quality is the epicentre of the company, where everyone feels responsible for the quality of what's delivered to the customer. In other companies, however, the narrative is blurred, which confuses teams as to what their responsibility to quality is and why it should matter to them.

Your company already has a quality narrative, because there is already a way in which people talk and think about quality in your company - even if you don’t think there is.

While this sounds similar to the “where do you want to go” section above, we tend to find that once teams have a clear vision of where they’re going, they are also sufficiently competent to work out a plan.

What we find is often missing, is the ability to articulate the strategy and resources needed in a way that gets the rest of the organisation to buy into the plan.

This requires you to create a new quality narrative, as creating the right quality narrative will influence and persuade others to get behind your plan.

How do you ensure you have the right quality narrative?

  • Understand other teams’ goals: To get the right quality narrative for your company, you need to convince others to buy into your plan. In order for them to buy into this, they need to know how this is going to help them to achieve their goals. If you know what they care about and their targets, you can align your vision in a way that assists them in attaining their goals.
  • Collect research and evidence: People look for external validation that something has worked elsewhere - use this to your benefit by showing relevant examples of how similar companies have prioritised quality and the upside for them. There’s also likely to be internal data within your organisation to support your cause, or personal accounts from others that can add credibility to your case.
  • Develop a plan and tell the story: After understanding each department’s goals and collecting your research, you can develop a plan and tell the story to whoever you need to. This involves describing the current situation and the consequences of not embracing a culture of quality, as well as the benefits of embracing this to them and the rest of the organisation.

When we worked with a rapidly scaling company, their quality lead had a hard time getting the engineering teams on board. He spoke with many engineers as well as others outside the organisation to figure out what engineers wanted, and these were the three areas that came up the most:

  • To create and release things that customers want
  • To be able to use the latest technology
  • To reduce time spent due to disruption (e.g. meetings or writing long documentation) to do the things they actually want to do

 

They then spoke with others in the organisation and other companies that they admired to collect research and create a story.

This enabled them to craft a message that would benefit engineering teams: by investing in quality from the beginning, there will be less re-work later down the line. This reduces the amount of context-switching they’ll have to do later down the line, which increases their productivity and enables them to focus on value-adding tasks.

As Dan Pink says in his book, To Sell is Human: “The average person nowadays spends 40% of their time convincing other people to do (or not do) things”.  Whether you are comfortable with the idea of storytelling and having to sell your ideas or not, this is what will help to ensure you have the right quality narrative.

What is a quality advocate and why do you need one?


A quality advocate’s role is to help promote quality across various parts of your organisation.

Why is having a quality advocate important?

Think about the below tweet:

It’s not sufficient to test solely for quality anymore - it has to be built in from the beginning. This can be done through a range of activities: clear and consistent acceptance criteria, ensuring adequate automated test coverage at the unit/integration level or encouraging collaboration, discussion on best practice for better testing throughout and more. This practice and sharing of knowledge encourages autonomy and transparency.

Atlassian, makers of well-known software products like Jira and Confluence, believe in the idea of "Quality Assistance”, which encourages the whole team to consider embedding quality into their workflow.

Shifting to a mindset of quality advocacy can help a company to embrace a culture of quality. However, you should be careful not to just shift more responsibility onto your development team by instructing them to simply write more tests or blocking them from releasing because everything has to be perfect.

However, if you're persistent and determined to see change, we’re sure that the above can guide you.  

Want more QAOps?

Drop your email in the box to be notified when we release the next instalment of the QAOps playbook.

Schedule a Demo