What is Usability Testing?

Usability testing is the practice of testing how easy a design is to use with a group of
representative users. It usually involves observing users as they attempt to complete tasks and
can be done for different types of designs. It is often conducted repeatedly, from early
development until a product’s release.

“It’s about catching customers

Usability Testing Leads to the Right Products

Through usability testing, you can find design flaws you might otherwise overlook. When you
watch how test users behave while they try to execute tasks, you’ll get vital insights into how
well your design/product works. Then, you can leverage these insights to make improvements.
Whenever you run a usability test, your chief objectives are to:

1) Determine whether testers can complete tasks successfully and independently.

2) Assess their performance and mental state as they try to complete tasks, to see how well
your design works.

3) See how much users enjoy using it.

4) Identify problems and their severity.

5) Find solutions.

While usability tests can help you create the right products, they shouldn’t be the only tool in
your UX research toolbox. If you just focus on the evaluation activity, you won’t improve the
usability overall.
There are different methods for usability testing. Which one you choose depends on your product
and where you are in your design process.

An Introduction to Usability Testing

Usability testing is a research method that helps you understand how well users interact with a
product in order to improve the overall experience. Traditionally, usability testing is used to
achieve the following goals:

1. Gain insight on the relative value of your product

In a competitive market, it’s important to understand what makes your solution better than the
other solutions people use. One of the goals of user testing, therefore, is to understand and
validate your solution’s unique value proposition. In other words, how does your product solve
problems better than existing solutions people commonly use?

2. Understand the most important features and why they are important

In addition to your product’s unique value proposition, it’s important to dig deeper to identify
features of your design that are the main drivers of value. This understanding will help you
clarify how much time and effort is required to further develop a particular aspect of your
product. Furthermore, understanding the tasks, features, and user flows that derive the most
value will guide you in creating more efficient, user-friendly designs.

3. Identify pain points in the user experience

Pain points are points of friction within the user experience that make a user’s interaction with
your product inefficient or unpleasant. Pain points are usually related to:

1. Navigation issues;
2. A users’ misunderstanding of your messaging or content (for websites and mobile apps);
3. A misalignment between what the user expects and what happens, or;
4. Experiences that keep the user from completing their goal in using your product.
Usability testing is conducted to help designers uncover these pain points and find ways
to redesign product features to overcome them.

4. Explore what people are doing with your solution and why

Lastly, and perhaps most importantly, usability testing will help you understand users’ actual
behaviors. It’s true that people often say one thing and do another. Therefore, it’s up to the user
researcher or designer to become an unbiased observer of the actual interactions that take place
between a user and a product. It’s also important to follow-up your observations with probing
questions that reveal the hidden motivations behind user behaviors.

Usability Testing Questions

Getting the information you need to design a product begins with asking the right questions.
Usability testing sessions can be divided into three main phases of questioning: pre-testing,
testing, and post-testing.

Pre-testing Questions
In this phase, the goal is to collect background information on the users engaging in the testing
session. Pre-testing questions should enable you to:

1. Uncover user demographics; 

2. Understand the existing approach to solving the problem your product aims to solve, and;
3. Explore the user’s prior knowledge with respect to your product and the domain in which
your solution exists.   

1. User Demographics

Understanding the user’s background and demographics is important to narrow down your target
market and identify factors that may affect use of your product. Example questions include:

 To get started, can you tell me briefly about yourself?

[Probe:] What is your current occupation?
[Probe:] What is your household composition?
 
 On a scale of 1 to 5 (1=not at all confident, 5=very confident), how would you rate your
level of confidence in using your mobile phone for [one or several tech savviness-related
activities]?

2. Existing Approaches

It’s also important to ask questions that give you a clear indication of what the ideal experience
would be like. This type of questioning provides insight on the relative benefits of your product.
Example questions include:

 How often do you [description of tasks related your product]?

[Probe:] When was the last time you engaged in this task?
[Probe:] What tools do you use, if any, to help with this task?
[Probe:] Please describe your experience with this tool.

3. Prior Knowledge as a Baseline

Lastly, it’s helpful to evaluate users’ awareness of the problem your product is solving. Prior
understanding can play a significant role in product usage. An example question includes:

 Which of the following indicates how much you know about [problem your product is
solving]?
[Option 1:] I don’t know anything about that.
[Option 2:] I know a little, but I could learn more
[Option 3:] I am an expert.

Testing Questions
In this phase, the goal is to allow the user to interact with your product, while uncovering pain
points within the experience. Testing questions should enable you to:

1. Identify why users engage or disengage with the product;

2. Understand how can you increase the product’s ease of use, and;
3. Explore how to increase users’ overall satisfaction with your product.

1. Observing User Behaviors

Defining specific tasks for users to achieve while using the product is critical to testing. In
addition, it’s important to observe the user’s actual interactions and how these interactions relate
to a pleasant or unpleasant user experience. Example questions include:

 What are you thinking as you view [specific page or feature]?

 If you were looking for [information], where would you expect to find it?
 How was the experience of using the product to complete this task?
[Probe:] What are your thoughts on the language used?
[Probe:] How easy or difficult was it to navigate?
[Probe:] What are your thoughts on the design and layout?
 [Internal] How long did it take the user to complete this task?

2. User Motivations for Behaviors

After observing user behaviors, it’s important to understand the “why” behind user interactions.
This feedback provides greater insight into what improvements should be made in order to align
your product with user expectations. Example questions include:

 Why did you navigate to [Page A] instead of [Page B]?

 What motivated you to click [a specific interaction]?

Post-Testing Questions

In this phase, the goal is to gather feedback on the overall user experience. Post-testing questions
should enable you to comprehensively evaluate the performance of each task, connected together
to form a seamless and cohesive experience.

1. End-to-End User Experience

An evaluation of the end-to-end user experience provides insight into any changes that need to
be made to the workflow within your product experience, such as a feature within an application
that feels disconnected from another. Example questions include:

 How would you describe your overall experience with the product?
 What did you like the most about using this product?
 What did you like the least?
 What, if anything, surprised you about the experience?
  What, if anything, caused you frustration?

2. Overall User Impression

Post-testing questions should also provide insight into the user’s impression with the product.
Example questions include:

 On a scale from 1 to 5 (1=not at all likely, 5=very likely), how likely are you to
recommend this product to a friend?
 How frequently would you use this product?
[Option 1:] Never
[Option 2:] Very Rarely (once per month)
[Option 3:] Rarely (2-3 times/month)
[Option 4:] Occasionally (2-3 times/week)
[Option 5:] Frequently (1-2 times/day)
[Option 6:] Very Frequently (3+ times/day)

Usability Testing Questions: Key Takeaways

As a UX designer, one of the most important skills is the ability to uncover user behaviors in
relation to your product. Usability testing is one method to place in your toolbox, and one of the
best ways to understand how real users experience your website or application. Check out come
of the resources below for more insight on how to approach your user testing sessions.

Do’s for usability testing questions: Uncover user flows and feelings

DO ask users to share what they are feeling and thinking of while interacting with a
product. Don’t focus only on the results of a particular task. Asking users to share their
comments will also help you gain insight into the goals of the user and their state of mind; this
may even surface other important insights that dramatically improve usability.

Example question: What are you feeling/thinking about when you complete this [action or task]?

DO give the destination and ask about the route. This kind of task analysis will help you
uncover the mental models and interaction patterns of your participants. Strive to find out as
much as possible about your users’ mental models as possible (how they complete a task). These
mental models will help you bridge the existing experience of your target audience with any
shortcomings (what’s hard or not enjoyable for them), and therefore you can decide on the next
iterations of your product and its features. You may create a super slick, elegant, minimalist
interface that looks harmonic but if users can’t find the menu, can’t search, or have no clear CTA
prompts, your product will not be truly successful.

Example question: Assuming you wanted to change the e-mail address associated with your
account, how would you go about doing that?
DO care about the facial expressions and body language of your users. We, humans, are
emotionally-driven creatures, so when testing a product, you should also observe and analyze the
non-verbal cues of participants. A word of caution: correlation is not causation. People may
demonstrate negative non-verbal signs not because of the product itself, but because of their
general state of mind or if they’re simply uncomfortable in their current situation/environment.

Don’ts for usability testing questions: A little less conversation, a little

more listening

Don’t provide too much guidance. Your goal is to make the testing session informative and
objective, so step back and avoid guiding users on how to respond.

Don’t prevent failures or missteps. If you observe a user carrying out a task in a way that is not
likely to result in the desired outcome, don’t interrupt, let them fail! For example, if a user-
initiated search is coming up empty, don’t say “Maybe you should try the filters for your
search?”. Our goal here is to better understand how and why the user fails, so we can improve.

Don’t help the user. On a similar note, you may face a situation where the user will explicitly
ask you what to do in a situation, but your goal is to encourage them to find the solution by
themselves (or declare it impossible, which sometimes happens). Don’t help them find a solution
as the results won’t be valuable in this case.

Don’t distract the user. Another mistake that’s easy to make is to interfere with the user’s
process when they’re already deep into an interaction. If you distract them, you’ll miss the
natural user flow.

Don’t talk too much. Even polite and friendly remarks or encouraging words may harm the
research process. Everything you say to the participant should correspond to the goal of the
testing. Don’t overload the participants with conversation; instead, let them direct their own
minds and focus on the task. You are observing how they interact with an experience, not
interviewing them.

Don’t ask yes/no questions. You don’t have unlimited time with participants. So, it’s much
handier to use your time with them to ask questions that give you lots of information, rather than
basic and uninformative yes or no questions. Ask open-ended questions that stimulate the users
and get them to describe exactly what they’re doing, feeling, or thinking.

Don’t believe everything your users say. Always leave room for a margin of error. Remember:
no matter how friendly, natural, and positive the testing atmosphere is, it is still not a natural
environment for people and that may influence their behavior and affect their responses.