Usability Testing -

The Complete Guide

Usability testing is a test method that involves assessing a website or application’s features by having end-users interact with the website or application

development-team-four@2x

Usability testing is all about getting real users to interact and test out your product so that you can improve your user experiences on the back of their targeted insights. This powerful evaluation tool is key to great web design and optimizing digital products so that they work at their most efficient and, most importantly, best serve users. 

We’ll be running you through everything you need to know about usability testing, from its key components to its most important metrics.

We’ll be discussing…

What is usability testing?

What is usability testing for?

Do I need usability testing?

Why is usability testing important?

Types of usability tests

How to execute usability testing

How to plan usability testing

Choosing and monitoring test metrics

Usability testing for success

Speak to a QA Expert

Speak to a QA Expert Launch a test for free

What is usability testing?

Usability testing is a specific type of iterative testing designed to assess how digital products, like websites or applications, function in the hands of real users. During a usability test participants are asked to try and complete set tasks using the digital product in order to reveal any areas where functionality or “usability” could be improved to benefit overall user experience. 

Participants will be testing for:

  • Overall effectiveness 
  • System efficiency 
  • Data accuracy 
  • User friendliness 

Usability studies posit questions like:

  • Is the system accessible and easy to learn (is it user-friendly)?
  • Does the system provide value for users?
  • Is the system design aesthetically pleasing?
  • Is the system intuitive and easy to navigate?
  • Are all screens/pages formatted uniformly?
  • Is the search function effective? 
  • Is system data up to date? 
  • Do all links work optimally?
  • Do users have access to support?

Usability testing vs. user testing

Usability testing is all too often confused with user testing. And it’s not hard to see why. These terms might seem interchangeable, but they actually refer to two quite different processes. 

To condense the difference into its most basic terms, usability testing is concerned with how people use a digital product, whereas user testing is about assessing whether people need the digital product in question. 

In other words, user testing is concerned with demand and usability testing assesses functionality. 

It’s also worth distinguishing usability testing methods from other UX research practices like A/B testing, questionnaires, and focus groups. Sure, they also assess user experience but none of those testing tools actually replicate the user experience directly with the collaboration of live participants. 

This is what makes usability testing unique and so important. 

What is usability testing for?

Usability testing is designed to identify potential problems that arise during the user’s experience of a digital product. Once these usability issues have been identified, designers can then set out to rectify the discrepancies and, once a product is deemed to be sufficiently usable, it can go to market. 

For example, if user researchers are trialing a new website, they will be asked to assess how easy that website is to use, navigate, and to make searches on. In other words, how easy was it for the participant to achieve their goal? 

Not only does usability testing improve the overall output of a digital product, it also makes sense economically. Companies save time and money by employing usability testing methods. Instead of fixing a product that has already been developed in full, design teams, and developers can make targeted fixes to any glitches that show up along the way. 

Usability testing in UX

Usability testing is a vital component for all UX (user experience) design processes. It allows developers to identify points of friction in the user journey and improve the overall usability of a new product. 

Reducing friction in the user experience isn’t just a point of perfectionism—it pays! 

The greater the friction working against your users, the more likely they are to look to competitor products and services. The lower the levels of friction, the more likely they are to stick around. Usability testing ensures that your user experiences are always smooth sailing and enticing, by taking into account a diverse range of perspectives and feedback from across your user base. 

UX is central to product success, and usability testing is the best way businesses and developers can make sure that their target user is at the core of every design decision. As such, usability testing is an essential phase in the development of any user experience design. This is what will ensure that your products offer features and functionalities that clients will pay for. 

Do I need usability testing?

We’ve established that usability testing is important, but what about who needs it, and when? There are many factors and criteria that might necessitate a business or developer to use usability testing as part of the developmental process. Let’s deep dive into a few of them. 

An excellent way to approach usability testing is to employ it across all stages of the user journey for a particular digital product. Usability testing should be conducted for features and processes that users will encounter during the discovery stage, early stage, mid stage, and late stage alike. Let’s frame this with an example. 

Say you’re launching a new redesign for your ecommerce website. You should aim to conduct usability testing on key webpages across the customer journey. 

  • An example of a discovery phase webpage for testing might be your expert profiles and testimonials page. Ensuring that these pages are optimized means that future visitors will be able to research and reach out to your experts with ease. 
  • An example of an early stage product for testing might be your ecommerce site’s accompanying mobile app. Once users have discovered your ecommerce brand, they’re likely (hopefully) to download your app and start browsing, so it's important to run a comprehensive mobile app test. If users have a seamless experience on the app, they’re going to be more likely to make their first purchase. 
  • An example of a mid stage webpage for testing might be your contact forms. Once a website visitor has generated enough interest in your brand, they might want to get in touch with a sales rep or sign up to your email newsletter. Ensuring that your contact forms are easy to navigate will encourage website visitors to proceed to this next stage. 
  • Finally, a late stage feature is something like your checkout page. Ensuring that your shopping cart and checkout processes are as user friendly as possible will reduce bounce and make sure all of those shopping baskets end in sales. 

All of the above examples present instances when usability testing would be beneficial. 

Why is usability testing important?

Usability testing reveals what real-life users really think about how a digital product works, looks, and functions. 

Real people reveal real issues that designers, marketers, and product developers might completely miss. When developing a new app or website, for example, it can be difficult to imagine how somebody with no prior knowledge of that product might experience it as a user.  

By engaging in usability testing with impartial participants it becomes easier to identify those features and functions that aren’t working optimally or that don’t suit the needs of your target market. 

Your participants should be able to assess whether or not your digital product is easy to understand, whether it delivers what they need, whether it works efficiently, and much more. In short, without robust usability testing, your UX designs risk falling at the first hurdle. 

Benefits of usability testing

Employing usability tests can be hugely beneficial for developers, as it’s a simple way to target every stage in the development process. Usability testing allows teams to test out functionality “in the field”, so to speak, and assess each new design/feature in real time. 

Some of the greatest benefits that come with effective usability testing include prototype validation, UX optimization, bug identification, and more. 

Let’s take a look. 

1. Prototype validation 


Usability testing is a great opportunity to validate your product prototype. During the early stages of development, once you’ve got your initial prototype ready, it can be really useful to bring in usability testers to see how they react to your design. 

Even though your product will still be in its infancy, the usability testers will be able to validate your concept and identify any issues early on (thereby saving you time and money in the long run). Testing your prototype in this way reduces your risk of needing to backtrack on your design or embark on costly redesigns later.

2. UX optimization

By conducting usability tests throughout the product development lifecycle, your completed product is more likely to work well first time round. You’ll have fixed all those navigation snags and user interface inefficiencies ahead of time. Because you’ve already identified any issues with your UI and flows, your product will be launched with sound and straightforward processes. 

3. Understanding the data

Chances are, you’re employing other testing methodologies alongside usability testing. That’s great. Usability testing is a fantastic accompaniment to other testing methods like heat mapping and A/B testing, for example. 

These other tests are great at quantifying data but often fail to get to the “why” behind a success or a problem. Usability testing gets straight to the source and qualifies the numbers through real user experiences.  

4. Identifying errors

Another important benefit of usability testing is that it is an excellent way to identify bugs before they become a bigger problem. Usability testers are great at picking up on the smaller errors in a product’s functionality, as well as the broader user experience as a whole.

For example, your test participants will be able to spot all those broken links, spelling errors, and search inefficiencies that might have passed you and your team by. These might seem like small, trivial things, but little errors like these can cause significant damage to brand reputation. 

Types of usability tests

The types of usability testing are best discussed using four key categories. These are:

  1. Remote usability testing vs. in-person testing
  2. Moderated vs. unmoderated testing
  3. Explorative vs. comparative testing 
  4. Summative testing 

The type of usability test that you need to employ will depend on a range of factors, including the resources you have available, your usability research objectives, and your target audience. 

Let’s take a look at each category in more detail below. 

1. Remote usability testing vs. in-person testing

Usability tests can be carried out in-person or remotely. Both methods are effective, although in-person testing is often more revealing, as it allows researchers to observe their participant’s non-verbal responses through body language, eye tracking, and expressions. 

That said, remote testing is a great substitute where in-person testing isn’t financially or practically feasible. It’s less time consuming, inexpensive, and you won’t have to pour resources into hiring a testing center. 

2. Moderated Vs. unmoderated testing

You’ll also need to decide whether you’re going to run moderated or unmoderated tests. 

Moderated test sessions are overseen by an experienced researcher. This can happen in-person or as part of a remote testing session. The moderator is responsible for introducing the test to your participants and answering any questions they may have. 

Unmoderated tests are unsupervised. In an unsupervised session, participants will trial your product either at a usability lab, or most commonly, in their own homes. While both methods have proven effective, moderated testing is better for generating deeper insights. 

Moderators can interact with participants and really get to know how they feel about the product—and why. Having said that, unmoderated testing is still better than no testing and can provide developers with a resource light and inexpensive way to gather usability insights from participants. 

3. Explorative vs. comparative testing

You’ll want to use either an explorative or a comparative testing template, depending on the type of information you’re trying to derive from your research participants. 

Explorative testing is essentially open-ended testing. In an explorative usability test scenario, participants will try out your product before being asked to give their opinions openly. By contrast, in a comparative testing scenario, participants will be given two products/designs and asked which they prefer. 

4. Summative testing

A summative testing environment (or assessment-based testing) involves observing how users engage and interact with your product through a task-based script.

As your participants work through each task, researchers can glean insights into how they navigate the product and identify any issues in the UX design that might be impeding the overall user experience. 

 

How to execute usability testing


Speak to a QA Expert Launch a test for free

To get the best results out of your usability testing, it’s important to execute the process carefully. Usability testing requires careful planning, clearly defined user tasks, participant recruitment, and appropriate levels of moderation. Each step is essential for optimizing outputs. 

Once you’ve decided on the type of usability testing framework you’re going to use, you’ll need to start planning your test. 

As part of your usability testing plan you’ll need to define any problems or areas to focus on during testing, identify the type of participants you want to involve in testing, and specify what questions you will ask them (what are you trying to find out?) 

Once planning is complete, it’s time to think about logistics. Things like where will you conduct the test? When (and for how long) will testing sessions run? Who will moderate the testing sessions and what skills and/or qualifications will you require of them? Will you be recording the testing session and, if so, what equipment will you need to prepare to make this happen? 

Finally, you’ll need to recruit your participants, define the tasks that they’ll need to participate in during the testing session, run the session, and organize all the data collected from the session in order to pull out targeted and actionable insights that will inform development. 

Key elements in usability testing

So, what are the most fundamental, key elements involved in usability testing? The answer is ...well...many. But there are a select number of core elements that are of utmost importance. 

These are (a) the tasks, (b) the facilitators, and (c) the test participants. 

Task

Tasks are the activities that your testing participants will perform during the usability testing session. These tasks should be designed to mimic what your future product users will do as they interact with your website or app. 

Whether you plan to deliver them verbally or on a task-sheet, make sure that your task instructions are extremely clear so that your participants can understand exactly what they’re being asked to do. One example of a task, for instance, could be to get participants to get rid of an error message on their screen. 

Facilitator

Facilitators guide participants through the testing process by giving them instructions (either verbally or on a written task-sheet), and answering any questions that participants may have during or after the testing process. 

The facilitator is a super important role because it ensures that tests run smoothly and that results are derived from valid, high-quality data. 

Participant

Needless to say, participants are the lifeblood of usability testing. But it’s important to select your participants wisely. This isn’t a situation where any Tom, Dick, or Harry off the street will do. 

In order to derive meaningful insights from your usability testing, you’ll need to select participants who would realistically be users of your type of product or service. Look for participants that have a similar background, interests, and needs as your end user or those who have already engaged with your products and services in the past. 

How to plan usability testing

Now we’ve got all the theory behind us, it’s time to start taking action. The very first thing you’ll want to do when embarking on a usability testing journey is to ace the planning phase. An awesome plan is the foundation of a great usability testing session and will ensure that you get the most out of your participants. 

Every usability testing study is going to be unique. No plan will be identical to another. Your plan should reflect your company’s individual goals, and take into account a realistic provision of resources, budgets, and logistical considerations. 

The most important components of any usability test plan include:

  1. Goal setting
  2. Format
  3. Sample size
  4. Recruit participants
  5. Task writing
  6. Pilot study 
  7. Metrics
  8. Write-up 

Let’s take a look at each of these components in more detail. 

1. Goal setting

Start your planning session by setting key goals. Meet with your stakeholders and figure out exactly what it is they want to discover from the research, then formulate appropriate questions, possible topics of concern, and an overarching goal for your research. 

Keep the number of goals you set to a minimum. Too many, and your testing will lose focus. Prioritize anything that is set to directly impact your product ROI and decide what data you’ll need in order to assess these factors. That data could be qualitative or quantitative data derived from your participants. 

2. Format

Next, turn your attention to how you will format your testing study. Remember those categories we discussed earlier? Now it’s time to think long and hard about which type of testing is most appropriate for your particular business and developmental objectives. 

Would your team members benefit more from in-person or remote testing? Does your testing require moderation, or can participants easily test your product solo from the comfort of their own home? 

3. Sample size

Thirdly, think about how many participants you’ll need in order to generate sufficient data for your research. Ideal sample sizes vary depending on a range of different criteria. 

For example, for qualitative studies, around five participants per user group is sufficient. However, if you’re assessing quantitative data, you’ll need to recruit more participants. Quantitative research requires larger sample sizes, typically ranging between 20 and 30 participants overall. 

4. Recruit participants

The next step in your planning process involves recruitment. Clearly your test is going to require participant involvement and you’ll need to ensure that you are recruiting representative participants for the job. By this, we mean participants who actually mirror your target market - those who could be real users. 

In order to find the best participants, you should start researching your target demographic ahead of time, so you can source individuals who match your target personas based on shared needs, goals, attitudes, user behaviors, and demographic data. 

5. Task writing

Plan, ahead of time, all the tasks that you’ll ask your participants to complete as part of the testing process. Make a list of activities that your participants will need to complete using your product. These activities should be designed in order to test out a range of different scenarios that your future users are likely to encounter when using your product. 

It can be beneficial for participants if tasks are written out on a task-sheet and framed in context (or as part of real-world scenarios). This will make the testing experience as close to real-life as possible. 

6. Pilot study 

If you want to be extra prepared, then it’s worthwhile conducting a pilot test before inviting your participants in for the real usability evaluation. 

Running a practice test will help you fine-tune your testing framework, tasks, and questions, so that the real usability test goes as smoothly as possible. Oftentimes final tweaks can be realized during the pilot. For example, you may find that you need to adjust your recruitment criteria or adjust some of the tasks on your task sheet. 

Choosing and monitoring test metrics

You’re nearly ready to launch your usability testing initiative. Congratulations!  A lot of planning and hard work has got you this far. The last thing to consider is how you’re going to monitor your testing session and the metrics that you’ll use to do so. 

There are three main categories when it comes to usability testing metrics. These are (1) effectiveness metrics, (2) efficiency metrics, and (3) satisfaction metrics. It’s best to use a combination of these three metric types. This will help you highlight any usability problems that arise in the user experience, across the board. 

Top metrics include:

Task success - % users that were able to complete tasks

Number of errors - Number of errors made by users whilst completing the task

Time on task - How long it takes participants to complete the task 

Single Ease Question (SEQ) - Perceived ease of completing the task on a scale of 1-7 

Subjective Mental Effort Question (SMEQ) - Participants rate difficulty of completing the task

Confidence - Participants rate how confident they felt completing the task on a scale of 1-7 

SUM - A single metric combining task success, ease, and time on task metrics 

Usability testing for success

Usability testing is a super important part of any digital product development process. Unlike other testing and some automated testing methods, usability testing is all about how real people use and interact with a product, enabling developers to glean targeted insights into their product's relative usability before it hits the open market. 

Need help with QA testing?

We’d love to give you a personal demo of our platform. Find out how we manage, execute and analyse test results to help you release high quality software anywhere in the world.

Ready? Let's talk