Draft

Edit or Request New Content

Usability Testing

What is it?

Participants are asked to complete one or more tasks while using a service. There are two types of usability testing sessions: moderated and unmoderated.

• Moderated. Throughout the test session, a moderator asks the participant to talk out loud while completing the tasks. In a separate space, the team watches the session and takes notes via video feed, if possible. A moderator is present throughout the session to guide the participant on what needs to be tested.
• Unmoderated. An unmoderated session is a series of tasks that are inputted into a testing system ahead of time. Then the participant is asked to go into the system to complete the tasks on their own. Although it is more convenient to plan compared to moderated testing, the downside of unmoderated testing is that if a participant gets stuck or does something unexpected, you won’t be able to probe in real time.

This guide will only cover moderated testing. Although it takes more time to plan and execute, it is beneficial for the following reasons

• You get the benefit of involving stakeholder and team throughout the session
• You can clarify and ask questions in real time
• You have time to ask follow up questions
• You have the flexibility to mix-and-match other design research techniques since you already have the participant present in the session
• If you have complex products that may not be straightforward to test, a moderator can observe and pivot quickly so there’s time to test other important aspects to test on

Purpose of usability testing

The purpose of usability testing is to put something in front of the participant and watch them work. This could be a prototype, a live service, or physical products.

Usability testing is about understanding a participant’s intention and goals when interacting with the service and to uncover any usability issues with the application.

Planning your tests

Leading up to your usability test sessions, you will need to consider:

Roles and responsibilities

Each member participating in design research will have specific roles and responsibilities.

User goals and tasks

To test if your participant interacts with the prototype the way it was designed to be, the participant will need instructions on a list of goals and tasks to complete while interacting with the prototype.

Privacy Tip: Consider a persona if you’re capturing information on the screen. If you plan to capture screen recordings and the system is asking for personal information, you may ask the participant to go through the system with fake data. Reason being that the focus is not on the accuracy of the participant’s information, but if the participant understands what the system is asking of them.

Number of participants to test with

The objective is to test up to a point where you can identify trends and patterns and further tests will not give you much new insight moving forward.

• Small sites and applications. Nielsen and Landauer (1993) report that the first five participants will uncover about 70% of major usability problems and next few participants will find nearly all the remaining problems.
• Information-rich sites. Five to eight participants may not uncover all of the usability problems. Many successful organizations have moved in the direction of continuous testing depending on budget and project needs
• Participants with different abilities. Test with at least one participant with physical or cognitive limitations or who needs assistance with digital applications. Testing with real people can uncover accessibility barriers better than a checklist and contributes to a more robust Gender Based Analysis+ process.

Lab Configuration or Test Location

To be very clear, usability testing doesn’t need a “lab”. You can run testing sessions almost anywhere, assuming you can set up the space properly. However, having a dedicated space helps make frequent testing easier and more seamless. For example, you will have better grasp on the quality of audio recordings, better control of schedules, and be in a space that will be familiar to you.

Ensure participants can access the building and the room for in-person testing sessions.

Ask beforehand if they need any special accommodations are needed prior to the test (for example, screen reader software or physical access).

Here are examples of how you can configure the space:

• One room setup. One room where participant uses the product with a moderator. Another room is dedicated for observers and notetakers to watch participant’s interaction with the product using recording equipment. If you can’t stream the test to another device, the notetaker will need to be in the room.
• Remote testing. Moderator, participant, and observers dial into a video conference call. Observers will remain quiet and watch what happens on the participant’s screen. The benefit of this setup is that you can test participants anywhere they would like to be situated. However, unstable internet connectivity may be an issue when conducting the tests.
• On site testing. You may also choose to conduct usability testing sessions in coffee shops, library, or other areas that is convenient to the participant as long as everyone feels comfortable with the situation. For on site testing sessions, do not go alone. It is always good to go with at least one of your teammates because you will need an extra helping hand and it will be easier to debrief with them after each session.

Type of Tasks

There are different types of tasks you may use to gather feedback from participants. When conducting usability testing sessions, the goal is to find out why your participant is asking the questions they are asking, rather than provide answers to their concerns.

For example, if a participant asks “what does it mean by…?” Rather than answering the question, you can reply “if you were to take a guess, what do you think it means?” This will help you get a good level of understanding based on their answer.

Verb-based Tasks

Verb-based tasks is when you ask participants to accomplish specific actions with the service. For example

• Upload your proof of registration
• Write a comment for review
• Pay the service fee

Scavenger Hunt Tasks

With scavenger-hunt tasks, you ask participants to find a specific piece of information. Scavenger hunt usually always begin with the verb “find” or are introduced with a scenario. For example

• Find the nearest ServiceBC office
• You recently moved from Alberta to British Columbia and need to change your health insurance coverage. Show me where you would go to find more information about this. What would you do next?

Interview-based Tasks

Before the participant arrives to the session, you won’t know what tasks the participant will be doing during the session. When the session starts, you will first conduct an interview with the participant to get a better idea on what information they will be looking for. You then give participants the opportunity to go to the service and accomplish their goals on the fly.

The best time to do interview-based tasks is when a website or service is populated with real information and data. Without real content and data for participants to explore, it will be hard to mirror their true experience.