Tips for Usability Testing with People with Disabilities

What is Usability Testing?

Usability testing (UT), a type of user experience (UX) research method, helps you gather quantitative and qualitative data from representative users. You can leverage UT to improve UX through iterative design based on the observations of real users. Once you identify interaction and design problems of a product or service, you can then test ways to solve those problems.

How Does Usability Testing Help?

UT is also a great way to gather valuable user insight, not only for the transactional nature of a system, but also for the process and its impact beyond the system tasks. Testing, which can occur in-person or remotely, should be an iterative of the design or development process before the release of a product or service.

A UT study usually involves observing participants while they perform tasks using a product or service, which helps validate design decisions and measure performance. UT that includes people with disabilities (PWD) can evaluate how well an Information and Communications Technology (ICT) product or service supports people with various impairments. Such testing can also help you ensure compliance with ICT accessibility statutes and applicable technical standards and create more inclusive and user-friendly products and services, such as improved accessibility, inclusive design, and a positive societal impact.

What Special Considerations are There?

When conducting UT with PWDs, you should consider several additional evaluation factors to account for diverse needs, such as keyboard accessibility, alternative text, responsive design, and color and contrast. It is important to adapt standard Usability Testing practices to accommodate PWDs and focus the UT study on accessibility issues. It is also important to also evaluate other factors for PWDs, including general usability, user satisfaction, plain language, and other similar criteria.

When exploring accessibility barriers, the session protocol may differ from general UT. For example:

You would likely use a think-aloud technique with significant facilitator interaction. You may need to adjust your use of this technique.
Data collection would focus on understanding errors related to accessibility barriers, rather than on time on task.
Tasks would look for areas of concern for potential accessibility problems, in addition to general system usage problems.

It is also important to evaluate other factors for PWDs, including general usability, user satisfaction, plain language, and other similar criteria.

How Can I Get Started?

While there are different ways to conduct testing with PWDs, an initial planning step of the UT study involves establishing goals and objectives. As in general UT, you must determine the key tasks participants will perform and what metrics to collect.

Next, identify the target user group, which should include a representative sample of users with a range of disabilities, such as vision, hearing, speech, dexterity, mobility, or cognitive disabilities. Many PWDs use customized software and devices, known as assistive technology, which include screen readers, braille readers, magnifiers, keyboards, or switches.

You will need to promote the usability study and conduct outreach to recruit your participants. It is important to allow enough time to announce your study, field queries from interested parties, and ensure that they meet your criteria. You should have a series of questions, called a screener, that will help you ensure that you have the representative sample that you need. Once participants are identified and then recruited, the test environment and the required facilitation materials should be accessible and compatible with participants’ needs.

During a study session, the facilitator will prompt participants through tasks, sharing scripted questions and task scenarios based on the understanding of the product or service being evaluated. Concurrently, the facilitator will observe and record participant behavior and feedback. Observers and notetakers may be virtually attending to record their observations as well for debriefing with the facilitator after each session.

Notes on Observations and Findings

It is up to the facilitator to determine the method of capture and analysis of findings. Facilitators must carefully consider all input and avoid assuming that input from one person with a disability applies to all PWDs. When analyzing the feedback from the UT session, the facilitator should evaluate user involvement with conformance to Section 508 Standards and other accessibility guidelines to identify a complete set of usability issues and opportunities for improvement. This means conducting the UT study once a product or service is ready for accessibility evaluation as part of the design and development schedule. The W3C Web Accessibility Initiative (WAI) provides examples of test objectives and specific accessibility barriers to look for.

Other considerations to help plan and conduct UT with PWDs include:

As part of the recruiting process, ask what assistive technology and version participants use. Participants may want to bring their own equipment to the testing location.
Make sure the testing location is accessible. Review the most accessible route and provide “know before you go” content for all necessary accommodations (accessible routes, service dog relief areas, etc.).
Plan to schedule additional time (roughly plan for 25% more time) for extra setup, and logistics.
If participants are using a screen reader, be sure not to talk over the screen reader during the UT session.
If the UT session is in person and a participant has a service animal, do not treat it as a pet. You may also need to allow time for the service animal to have water and building entry and exit for relief breaks.
Protect the test participants’ rights and respect them as experts. Let them verbalize their experience and track that data. Facilitators should not worry about using the word “see” as in “did you see that on the web site?” You should, however, aim to be considerate by being responsive to individual needs and preferences and simply asking individuals if you aren’t sure.

Final Thoughts

When you incorporate accessibility standards throughout the development lifecycle of any product or service, you not only improve compliance to legal requirements but also improve accessibility for all users.
By including users who have various types of impairments in UT studies, you enhance the user experience and satisfaction of people who often face technological barriers to information.
You should conduct UT with PWDs to encourage inclusion of all users, regardless of their abilities.

Additional Resources for Usability Testing with People with Disabilities

Reviewed/Updated: September 2023

Content Contributors:

Beth Martin

Contributing Organizations

Office of Personnel Management