Usability testing is a test method that involves assessing a website or application’s features by having end-users interact with the website or application
Usability testing is all about getting real users to interact and test out your product so that you can improve your user experiences on the back of their targeted insights. This powerful evaluation tool is key to great web design and optimizing digital products so that they work at their most efficient and, most importantly, best serve users.
We’ll be running you through everything you need to know about usability testing, from its key components to its most important metrics.
We’ll be discussing…
Usability testing is a specific type of iterative testing designed to assess how digital products, like websites or applications, function in the hands of real users. During a usability test participants are asked to try and complete set tasks using the digital product in order to reveal any areas where functionality or “usability” could be improved to benefit overall user experience.
Participants will be testing for:
Usability studies posit questions like:
To get the best results out of your usability testing, it’s important to execute the process carefully. Usability testing requires careful planning, clearly defined user tasks, participant recruitment, and appropriate levels of moderation. Each step is essential for optimizing outputs.
Once you’ve decided on the type of usability testing framework you’re going to use, you’ll need to start planning your test.
As part of your usability testing plan you’ll need to define any problems or areas to focus on during testing, identify the type of participants you want to involve in testing, and specify what questions you will ask them (what are you trying to find out?)
Once planning is complete, it’s time to think about logistics. Things like where will you conduct the test? When (and for how long) will testing sessions run? Who will moderate the testing sessions and what skills and/or qualifications will you require of them? Will you be recording the testing session and, if so, what equipment will you need to prepare to make this happen?
Finally, you’ll need to recruit your participants, define the tasks that they’ll need to participate in during the testing session, run the session, and organize all the data collected from the session in order to pull out targeted and actionable insights that will inform development.
So, what are the most fundamental, key elements involved in usability testing? The answer is ...well...many. But there are a select number of core elements that are of utmost importance.
These are (a) the tasks, (b) the facilitators, and (c) the test participants.
Tasks are the activities that your testing participants will perform during the usability testing session. These tasks should be designed to mimic what your future product users will do as they interact with your website or app.
Whether you plan to deliver them verbally or on a task-sheet, make sure that your task instructions are extremely clear so that your participants can understand exactly what they’re being asked to do. One example of a task, for instance, could be to get participants to get rid of an error message on their screen.
Facilitators guide participants through the testing process by giving them instructions (either verbally or on a written task-sheet), and answering any questions that participants may have during or after the testing process.
The facilitator is a super important role because it ensures that tests run smoothly and that results are derived from valid, high-quality data.
Needless to say, participants are the lifeblood of usability testing. But it’s important to select your participants wisely. This isn’t a situation where any Tom, Dick, or Harry off the street will do.
In order to derive meaningful insights from your usability testing, you’ll need to select participants who would realistically be users of your type of product or service. Look for participants that have a similar background, interests, and needs as your end user or those who have already engaged with your products and services in the past.
Now we’ve got all the theory behind us, it’s time to start taking action. The very first thing you’ll want to do when embarking on a usability testing journey is to ace the planning phase. An awesome plan is the foundation of a great usability testing session and will ensure that you get the most out of your participants.
Every usability testing study is going to be unique. No plan will be identical to another. Your plan should reflect your company’s individual goals, and take into account a realistic provision of resources, budgets, and logistical considerations.
The most important components of any usability test plan include:
Let’s take a look at each of these components in more detail.
Start your planning session by setting key goals. Meet with your stakeholders and figure out exactly what it is they want to discover from the research, then formulate appropriate questions, possible topics of concern, and an overarching goal for your research.
Keep the number of goals you set to a minimum. Too many, and your testing will lose focus. Prioritize anything that is set to directly impact your product ROI and decide what data you’ll need in order to assess these factors. That data could be qualitative or quantitative data derived from your participants.
Next, turn your attention to how you will format your testing study. Remember those categories we discussed earlier? Now it’s time to think long and hard about which type of testing is most appropriate for your particular business and developmental objectives.
Would your team members benefit more from in-person or remote testing? Does your testing require moderation, or can participants easily test your product solo from the comfort of their own home?
Thirdly, think about how many participants you’ll need in order to generate sufficient data for your research. Ideal sample sizes vary depending on a range of different criteria.
For example, for qualitative studies, around five participants per user group is sufficient. However, if you’re assessing quantitative data, you’ll need to recruit more participants. Quantitative research requires larger sample sizes, typically ranging between 20 and 30 participants overall.
The next step in your planning process involves recruitment. Clearly your test is going to require participant involvement and you’ll need to ensure that you are recruiting representative participants for the job. By this, we mean participants who actually mirror your target market - those who could be real users.
In order to find the best participants, you should start researching your target demographic ahead of time, so you can source individuals who match your target personas based on shared needs, goals, attitudes, user behaviors, and demographic data.
Plan, ahead of time, all the tasks that you’ll ask your participants to complete as part of the testing process. Make a list of activities that your participants will need to complete using your product. These activities should be designed in order to test out a range of different scenarios that your future users are likely to encounter when using your product.
It can be beneficial for participants if tasks are written out on a task-sheet and framed in context (or as part of real-world scenarios). This will make the testing experience as close to real-life as possible.
If you want to be extra prepared, then it’s worthwhile conducting a pilot test before inviting your participants in for the real usability evaluation.
Running a practice test will help you fine-tune your testing framework, tasks, and questions, so that the real usability test goes as smoothly as possible. Oftentimes final tweaks can be realized during the pilot. For example, you may find that you need to adjust your recruitment criteria or adjust some of the tasks on your task sheet.
You’re nearly ready to launch your usability testing initiative. Congratulations! A lot of planning and hard work has got you this far. The last thing to consider is how you’re going to monitor your testing session and the metrics that you’ll use to do so.
There are three main categories when it comes to usability testing metrics. These are (1) effectiveness metrics, (2) efficiency metrics, and (3) satisfaction metrics. It’s best to use a combination of these three metric types. This will help you highlight any usability problems that arise in the user experience, across the board.
Top metrics include:
Task success - % users that were able to complete tasks
Number of errors - Number of errors made by users whilst completing the task
Time on task - How long it takes participants to complete the task
Single Ease Question (SEQ) - Perceived ease of completing the task on a scale of 1-7
Subjective Mental Effort Question (SMEQ) - Participants rate difficulty of completing the task
Confidence - Participants rate how confident they felt completing the task on a scale of 1-7
SUM - A single metric combining task success, ease, and time on task metrics
Usability testing is a super important part of any digital product development process. Unlike other testing and some automated testing methods, usability testing is all about how real people use and interact with a product, enabling developers to glean targeted insights into their product's relative usability before it hits the open market.