AB

is
v
Ad
Q
Usability Engineering
according to IEC 62366-1
22
20
Training 2022-09-27
©
Schedule (CET)

AB
8:30 Start

is
v
Ad
10:00 – 10:15 Fika break

Q
212:00 – 13:00
2 Lunch break
20

14:00 – 14:15 Fika break

©

15:30 End
Agenda

AB
Introduction, presentation of participants

v is
Ad
1. Regulations, Standards and Guidances

Q
2. Overview of IEC 62366-1 & Definitions
22
20

3. The Usability Engineering Process

©

4. Legacy Devices - UOUP

Presentation of the speaker – Cilla Lundevall

AB
• Senior Quality and Regulatory

is
Consultant @ QAdvis

v
Ad
• MSc EE & Medical Radiation physics

Q
• 25 years in Medical Device Industry
22
• 20+ years in Medical Device QA
20

– V&V
– QA/RA Consultant
©

– PRRC, Head of QA/RA

– SEK TK62
Presentation of participants

AB
• Name

vis
• Company

Ad
• Role

Q
• Kind of products
2 2
20

• Expectations on this training

©
©
20
22
Q
Ad
vis
AB
Usability and Medical Devices
A recent example on recall due to poor usability design (May 2018)

AB
• Recall of Jaundice Meters used to measure bilirubin in
newborn infants

is
Jaundice Meters Recalled Due to Users have misinterpreted display messages that have
resulted in serious injuries

v
Misinterpretation of Display Messages

Ad
for Out of Range Values
• Display shows (- - -) or (-0-) when the bilirubin level is
higher than the maximum level of detection

Q
Misinterpreted as low or zero
(- - -) (-0-)
2
•
2 The reading is not intuitive or clear
20
The misuse of display information could increase risk
for serious adverse health consequences such as
acute and/or chronic brain damage or death
©

Ref: https://www.fda.gov/MedicalDevices/Safety/ListofRecalls/ucm611564.htm
Usability and Medical Devices
Some examples from UPAXA (The User Experience Professionals Association)

AB
• Glucose meter recall – poor visibility decimal
point

v is
Ad
• Monitoring device recall - ineffective alarm
system

Q
2 2
• “In 2008, the FDA recalled an ultrasound
20
system because the graphics made users
misunderstand the image orientations of the
©

patient’s left and right sides”

Ref: http://uxpamagazine.org
 Usability and Medical Devices
A recent example on recall due to poor information for safety (March 2022)

AB
• Recall of system for support of placement of
medical feeding tubes

v is
• Users have misplaced tubes when using the

Ad
system, causing 60 patient injuries and 23
deaths since 2015

Q
2 •2 Users neglects to confirm tube placement based
on their institution's protocols before using the
20
tube to deliver nutrition
©

• The device labelling is not clear enough with the

need for confirmation of placement of the
feeding tube
Ref: https://www.fda.gov/medical-devices/medical-device-recalls/avanos-medical-recalls-cortrak2-enteral-access-system-risk-misplaced-enteral-tubes-could-
cause
 Usability and Medical Devices
A recent example on recall due to poor information for safety (April 2022)

AB
• Recall of system for respiratory aid, for home-
use

v is
• The in-line ventilator adaptor may prevent home-

Ad
use patients from getting enough oxygen from
their ventilators. One injury and two deaths

Q
reported.
2
•
2 The risk of serious injury or death is more
20
significant in home-care settings if the
caregivers have not been trained properly, the
device is not connected properly, or if caregiver
©

is not prepared to properly address issues with

the system
Ref: https://www.fda.gov/medical-devices/medical-device-recalls/baxter-healthcare-corporation-recalls-volara-system-risk-respiratory-distress-ventilated-
patients
The Importance of Usability

AB
Use errors caused by As healthcare evolves,
inadequate medical device usability less skilled users

is
have become an including patients themselves
are now using medical devices

v
increasing cause for concern
and the devices are becoming

Ad
more complicated

Q
Many of the medical devices developed
without applying a usability engineering
process are non-intuitive
2 2 The design of the user interface
difficult to learn
20
to achieve adequate usability
difficult to use requires a different process and skill set
than that of the technical implementation
©

of the user interface

From IEC 62366-1:2015 + AMD1:2020

 AB
isv
Ad
Q
REGULATIONS, STANDARDS &
GUIDANCES
22
20
©
Regulatory Context

AB
MDD

is
Medical Device Directive
MDR

v
Ad
AIMDD Medical Device Regulation

Q
Active Implantable
Medical Device Directive
2 2
20

IVDD IVDR
©

In Vitro Diagnostic In Vitro Diagnostic Medical

Medical Device Directive Device Regulation
 Standards and Guidance Documents

AB
Guidance

is
EU Commission National Laws Standards

v
Documents

Ad
Legally Not legally

Q
binding binding
2 2
20

MDR IVDR IEC 62366-1

©

MDR = the Medical Device Regulation (Regulation 2017/745)

IVDR = the In Vitro Medical Device Regulation (Regulation 2017/746)
Standards and Guidance Documents

AB
IEC 62366-1:2015/A1:2020 Not harmonized in EU
Application of usability engineering to medical devices Recognized as consensus standard by the FDA
ANSI/AAMI/IEC 62366-1:2015

is
IEC TR 62366-2:2016 Guidance to IEC 62366-1:2015

v
Guidance on the application of usability engineering to medical devices

Ad
EN 62366:2008 Harmonized standard vs EC Directives 93/42/EEC (MDD) and
Medical devices - Application of usability engineering to medical devices 98/79/EC (IVDD)

Q
EN 60601-1-6:2010 Harmonized standard vs EC Directives 93/42/EEC (MDD) and
Medical electrical equipment -- Part 1-6: General requirements for basic 90/385/EEC (AIMDD)
safety and essential performance - Collateral standard: Usability Refers to EN 62366:2008
FDA Guidance 2016
2 2
Applying human factors and usability engineering to medical devices
Guidance on usability engineering
20

ANSI/AAMI/HE75:2009/(R)2013 Comprehensive guidance on usability engineering

Human factors engineering – design of medical devices Recognized as consensus standard by the FDA
©

IEC = The International Electrotechnical Commission

ANSI = The American National Standards Institute
AAMI = Association for the Advancement of Medical Instrumentation
Additional FDA Guidance

AB
is
List of device types that should have usability data

v
included in premarket submissions, i.e. PMA and

Ad
510(k)

Q
22
20
©
 Additional FDA Guidance – Devices Listed

AB
Ablation generators Gastroenterology-urology Negative-pressure wound
endoscopic ultrasound systems therapy intended for home

is
Anesthesia machines with elevator channels use

v
Ad
Artificial pancreas systems Hemodialysis and peritoneal Robotic catheter
dialysis systems manipulation systems

Q
Auto injectors
Implanted infusion pumps Robotic surgery devices
Automated external
2 2
Infusion pumps
defibrillators Ventilators
20

Insulin delivery systems

Duodenoscopes with elevator Ventricular assist devices
©

channels
Additional FDA Guidance

AB
• For devices on the list:

is
– Usability data are required

v
– The usability engineering process and the results

Ad
of usability testing are provided in the Usability
Engineering Summary

Q
22
• For devices not on the list:
20
– Usability data should be submitted if a use error
could result in serious harm
©

– Usability data may be requested on a case-by-

case basis (for examples, see guidance
document)
Standards and Guidance Documents...

AB
is
GSPR 5 are legally non-binding…

v
Ad
MDR – 2017/745

Q
IVDR – 2017/746 2
…but to be in compliance with MDR / IVDR
2
usability has to be considered
20
©

GSPR = General Safety and Performance Requirement

MDR / IVDR Requirements on Usability – GSPR 5

AB
In eliminating or reducing risks related to use error,
the manufacturer shall

v is
Ad
Design for patient safety Design for lay, professional,
disabled or other users

Q
2
Reduce as far as possible the
2
technical knowledge experience education
20
risks related to the ergonomic
features of the device and the
©

environment in which the device training medical and physical conditions

is intended to be used

use environment
GSPR = General Safety and Performance Requirement
Therapeutic Goods (Medical Devices) Regulation 2002
Medical devices Essential Principles

AB
is
• 10 - Medical devices with a measuring function:

v
…

Ad
(2) The measurement, monitoring and display scale of the device

Q
must be designed and produced in accordance with ergonomic
2
principles, having regard to the intended purpose of the device.
2
20
©
Therapeutic Goods (Medical Devices) Regulation 2002
Medical devices Essential Principles

AB
• 12.1 - Programmed or programmable medical device or

v is
software that is a medical device

Ad
(1) A programmed or programmable medical device, or software that

Q
is a medical device, that is intended to make use of either or both of
2
data and information must be designed and produced in a way that
2
ensures that: (a) the safety, performance, reliability, accuracy,
20

precision, useability, security and repeatability of the device are

©

appropriate for the intended purpose of the device;

 AB
• Why focus on usability?

is
– Develop a safe and usable product

v
Ad
– CE marking requires compliance with MDR

Q
22
• Why follow a standard?
20

– A systematic approach facilitates the development

of a safe product
©

– Facilitates review of technical documentation

 AB
vis
Ad
Q
OVERVIEW OF 62366-1
& DEFINITIONS 2 2
20
©
 Application of Usability Engineering to Medical Devices

EN 62366:2008 IEC 62366-1:2015

AB
 Reflecting a greater awareness of the

is
importance of usability for device safety Many of the medical devices developed

v
without applying a usability engineering

Ad
process are non-intuitive
difficult to learn
Use errors caused by difficult to use

Q
inadequate medical device usability
have become an
increasing cause for concern
2 2 As healthcare evolves,
less skilled users
20
including patients themselves
are now using medical devices
and the devices are becoming
©

more complicated
 Application of Usability Engineering to Medical Devices

AB
EN 62366:2008 IEC 62366-1:2015 IEC 62366-1:2015 + AMD1:2020

is
 Reflecting a greater awareness of the  Harmonisation with EN ISO 14971:2019

v
importance of usability for device safety

Ad
 Closer association with EN ISO 14971  New requirements on summative usability test plan
(Application of risk management to  Intended user profiles vs test participants justification

Q
medical devices)  User groups vs number of participants justification
 Intended use environment vs test environment justification
 Harmonization with FDA expectations
2 2  Definition of correct use for each hazard-related
use scenario
20
 Implementation of common practices
(e.g. formative and summative testing)
 Clarification of terminology
©

 Divided into standard text (what to do) Primary operating function - Critical task - Close call
and guidance (how to do it)
 AB
is
Usability

v
Ad
IEC 62366-1:2015 +AMD1:2020
IEC TR 62366-2:2016

Q
2 2
20
©
62366-1 vs 62366-2

AB
IEC 62366-1:2015 + AMD1:2020 IEC TR 62366-2:2016
International standard Guidance (no regulatory purpose)

v is
Content • Description of the usability • Guidance on how to implement a

Ad
engineering process usability engineering process
• Examples on usability engineering

Q
methods

Scope • Safety
2 2 • Safety
• Commercial aspects
20

(task efficiency, user satisfaction)

©

Mapping between part 1 and part 2 is available (table 1 in part 2)

IEC 62366-1:2015 + AMD1:2020

Focus on Safety

AB
is
Safety

v
Manufacturers can choose to implement

Ad
a usability engineering program focused
Safety or + narrowly on safety or more broadly on

Q
safety and other attributes
Other 2 2
aspects
20
©

IEC 62366-1:2015 + AMD1:2020

 IEC 62366-1 Focuses on Safety

AB
IEC 62366-1:2015 + IEC TR 62366-2:2016
Safety AMD1:2020 Guidance (no regulatory

is
International standard purpose)

v
Safety +

Ad
or Application of a usability Guidance on how to implement
engineering process to a usability engineering process,
Other

Q
optimise medical device incl. examples on usability
aspects usability engineering methods
2 2
20
Safety Safety
Manufacturers can choose to implement
Commercial aspects (task
a usability engineering program focused
efficiency, user satisfaction)
©

narrowly on safety or more broadly on

safety and other attributes
Mapping between part 1 and part 2 is available (table 1, part 2)
IEC 62366-1:2015 + AMD1:2020
Definitions

AB
User Person interacting with the medical device

is
E.g. nurses, physicians, laboratory technicians,

v
patients, service personnel

Ad
Q
Usability Characteristic of the user interface
that facilitates use and thereby
22 establishes effectiveness, efficiency and
user satisfaction in the intended use
20
environment
©

effectiveness = accuracy and completeness with which users achieve specified goals
efficiency = resources expended in relation to effectiveness

IEC 62366-1:2015 + AMD1:2020

Definitions

AB
is
• No absolute consensus on terminology
Usability engineering / Human factors engineering

v
The application of knowledge about human

Ad
behaviour, abilities, limitations, and other • 62366-1 and FDA uses the terms
characteristics related to the design of interchangeably

Q
medical devices (including software), systems
and tasks to achieve adequate usability
2 2
20
©

IEC 62366-1:2015 + AMD1:2020

User Interface

AB
• All means of interaction between the user

is
Action Input and the medical device

v
Ad
• Includes e.g. buttons, touch screens,
mechanical parts, handles, alarms,

Q
Cognition Processing accompanying documents…
2
2
20

Perception Output
©

Human Interface Device

Use error
Hazardous
situation

AB
is
Action Input

v
Ad
Q
Cognition Processing
2
Cognition error
2
20

Hazardous
Perception Output
situation
©

Perception error

Human Interface Device

Definitions – normal/abnormal use, use error

AB
Normal use

is
Correct use
Following good practice

v
Ad
vs
Use error: User action or lack of user action

Q
while using the medical device that leads to a
vs different result than that intended by the
Use error
22 manufacturer or expected by the user.
20

Abnormal use non-qualified person

©

Contrary to safety handling against contraindication

information reckless use

IEC 62366-1:2015 + AMD1:2020

 Adapted from IEC 62366-1:2015 + AMD1:2020

Usability Engineering vs Risk Management

AB
Scope of ISO 14971
Scope of IEC 62366-1

is
Reasonably

v
foreseeable use Normal use Correct use

Ad
Q
Use
Use error
2 2
20

Abnormal use
©

Not reasonably Reasonably foreseeable misuse

foreseeable use as defined in ISO 14971
 AB
isv
Ad
Q
THE USABILITY ENGINEERING
PROCESS
22
20
©
The Usability Engineering Process – General

AB
• The manufacturer shall establish, document and
maintain a Usability Engineering Process to provide

is
safety for the patient, users and others

v
Ad
• The process shall address user interactions with the
medical device according to the accompanying

Q
document, including, but not limited to:
22 •
•
Transport
Storage
20

• Installation
• Operation
©

• Maintenance & repair

• Disposal

IEC 62366-1:2015 + AMD1:2020

Tailoring of the Usability Engineering Process

AB
Level of effort and choice of methods may vary based on:

is
• Size and complexity of user interface

v
• Severity of harm associated with use

Ad
• Extent or complexity of use specification

Q
• Extent of device modification to an existing device
2 2
20
• Presence of User Interface of Unknown Provenance (UOUP)
©
The Usability Engineering File

AB
• The results of the Usability Engineering
Process shall be stored in the Usability

is
Engineering File

v
Ad
• The records and other documents that form

Q
the Usability Engineering File may form part
22 of other documents and files
20
©

-> See content in

Usability Engineering File_Contents.pdf

IEC 62366-1:2015 + AMD1:2020

 Medical Devices – Examples on Documentation

AB
C D

B E

is
Use
...
scenarios

v
Ad
A Technical F ... Formal
File testing

Q
... ... 2 2 Use
Specification ...
20
Analysis ...
Usability
Engineering
©

Plan
Risk Summary
File
Mgmt
File
Usability – an Iterative Process

AB
is
Refined

v
User input User input ”Final”
concept

Ad
product

Q
Early
concept 2 CE mark
2
20

User input User input

©

Improved
Idea product
Usability – Closely Related to Risk Management Process

B, C, D, F: Information flow required by 62366-1:2015 + AMD1:2020

AB
Identify user interface

is
B
Identify characteristics characteristics related to safety

v
related to safety and potential use errors

Ad
Identify known or foreseeable

Q
C
hazards and hazardous situations
Identify hazards and
2 2hazardous situations
Identify and describe hazard-
D
20
related use scenarios
©

E Further improvement not

Is residual risk acceptable?
necessary and practicable

14971:2019 62366-1:2015 + AMD1:2020 14971:2019 62366-1:2015 + AMD1:2020

The Usability Engineering Process - Components
Define important characteristics

AB
Investigate risks and specify the
User interface

v is
Ad
Q
2 Design & development
2
20
©

Validate and evaluate residual risk

The Usability Engineering Process

1. Use specification

AB
2. User interface characteristics related to
safety and potential use errors

is
3. Known or foreseeable hazards and

v
hazardous situations

Ad
4. Hazard-related use scenarios + selection for
summative testing

Q
5. User interface specification
22 6. User interface evaluation plan
(formative and summative evaluations)
20

7. User interface design incl. formative

evaluation
©

8. Summative evaluation
Use Specification

AB
Headings 62366-1:2015 + AMD1:2020

is
Intended medical indication

v
Ad
Intended patient population

Intended part of the body or type of Input to Intended Use

Q
tissue applied to or interacted with

Intended user profile

2 2
20

Intended use environment IEC TR 62366-2:2016

8.4: Examples on methods for developing the use specification
©

Operating principle Annex G: Example on description of user profile

Annex H: Example on description of use environment
Use Specification - Extent

AB
is
• Risk profile (patient, user)

v
• Degree of human interaction

Ad
Q
22
20
©
Use Specification - Examples

AB
Headings 62366-1 Example – Intracardiac Blood Example – SW for Histological
Pressure Measurement Device Grading of Melanoma

is
Intended medical indication Intracardiac blood pressure Melanoma

v
measurement

Ad
Intended patient population Adults in need of heart Patients with suspected or
catheterisation diagnosed melanoma

Q
Intended part of the body or Vascular system and cardiac atrias N/A
type of tissue applied to or
interacted with
2
and ventricles
2
20
Intended user profile Interventional cardiologist / Laboratory technicians and
cardiac surgeon physicians
©

Intended use environment Operation theater / room for Pathology department (laboratory)
interventional cardiology
Operating principle Optical pressure sensor Software algorithms
 Personas – User Profile

AB
Anna, 84 years, widow, John, 24 years, university Andrew, 48 years, nurse,
living alone in an apartment, student (mathematics), head of diabetes clinic at

is
home care once a day amateur musician Smallford hospital

v
Ad
• “Real-world consideration”

• Express the major needs

Q
”Everyone should get the
”I am a little afraid of all this ”I refuse to let the disease
best possible care at our
and expectations of the
new technology.”
2 2
control my life.”
hospital.” most important user groups
20
Goals Goals Goals • Aid in prioritizing features
To meet my friends often To live an independent life To help as many patients as
fast as possible
©

To avoid going to hospital To ’always’ use my mobile

phone To diagnose patients at
To take care of myself home through smart devices
To count statistics on
everything
Exercise #1: Use Specification

AB
• Choose your own product
or https://www.medicalnewstoday.com/articles/326663

is
• Write a Use Specification, including the components

v
specified by the standard

Ad
• 15 min

Q
•
2 E-mail to cilla.lundevall@qadvis.com
2
20
• Basis for course certificate

• Description of the exercise is available in the Dropbox

©

(Exercise #1)
The Usability Engineering Process

1. Use specification

AB
2. User interface characteristics related to
safety and potential use errors

is
3. Known or foreseeable hazards and

v
hazardous situations

Ad
4. Hazard-related use scenarios + selection for
summative testing

Q
5. User interface specification
22 6. User interface evaluation plan
(formative and summative evaluations)
20

7. User interface design incl. formative

evaluation
©

8. Summative evaluation
User Interface Characteristics Related to Safety and
Potential Use Errors

AB
B Identify user interface
Identify characteristics

is
characteristics related to safety
related to safety

v
and potential use errors

Ad
14971:2019 62366-1:2015 + AMD1:2020

Q
Input
2 2
20
• Annex A ISO/TR 24971:2020 A.2.31 • Task analysis - a tool to identify potential use
Questions for identification of hazards and errors (IEC TR 62366-2:2016, 9.1)
medical device characteristics related to
©

safety. Includes usability-related aspects. • Primary Operating functions (if provided in

applicable particular product specific standard)
• Experience from previous products
(customer complaints, incidents, literature)
Characteristics related to the User interface

AB
• UI design features
• Distracting environment

is
• Connecting parts/accessories

v
Ad
• Control interface
• Displayed information

Q
• Controlled by a menu
2 2
• Successful use dependent on user knowledge, skills and
20

abilities
• Will it be used by users with specific needs?
©

• Restricted/authorized access
Annex A ISO/TR 24971:2020 A.2.31
Other usability characteristics

AB
• Alarm system

is
• Misuse

v
Ad
• Mobility/portability

Q
• Essential performance (60601)
• Degree of autonomy
2 2
20

• Product output – how to use?

©

Annex A ISO/TR 24971:2020 A.2.32-A.2.37

Primary Operating Functions - Example

AB
SS-EN 60601-2-12 Critical care ventilators

is
1. setting the healthcare professional operator-adjustable controls:

v
i. setting alarm limits;
ii. inactivating alarm signals;

Ad
iii. switching between different ventilation-modes and inflation-types; and
iv. setting ventilation control parameters;

Q
EXAMPLE 1 Tidal volume, set rate, Baseline Airway Pressure, pressure support

7. 2
connecting or disconnecting the patient-connection port of the Ventilator
2
Breathing System to the patient-interface;
20
8. starting the ventilator from power off;
9. turning off the ventilator;
10. performing a basic pre-use functional check of the ventilator including the
©

alarm system;
11. processing the ventilator between patient uses.
Exercise #2: Characteristics related to safety

AB
• Chose your own product
or

is
https://www.medicalnewstoday.com/articles/3

v
26663

Ad
• Identify characteristics related to safety for

Q
this product
22
20
• 5 min to study the product, 15 min for
identifying characteristics related to safety
©
The Usability Engineering Process

1. Use specification

AB
2. User interface characteristics related to
safety and potential use errors

is
3. Known or foreseeable hazards and

v
hazardous situations

Ad
4. Hazard-related use scenarios + selection for
summative testing

Q
5. User interface specification
22 6. User interface evaluation plan
(formative and summative evaluations)
20

7. User interface design incl. formative

evaluation
©

8. Summative evaluation
Known or Foreseeable Hazards and Hazardous
Situations

AB
• Part of risk management process according to
ISO 14971:2019

v is
• Identification of hazardous situations and harms related

Ad
to potential use errors

Q
• Use specification to be considered
22
• Separate usability risk assessment or included in
20
general risk assessment
©
Hazardous situations
Hazardous
situation

AB
is
Action Input

v
Ad
Q
Cognition Processing
2
Cognition error
2
20

Hazardous
Perception Output
situation
©

Perception error

Human Interface Device

Questions to Consider when Assessing Use-associated
Risks

AB
• Will the user be able to locate the desired function/operation?
• Will the user understand how to initiate the desired function/operation?

v is
Ad
Alarm
Start Calibration

Q
settings
2 2
20
37 19
©

0.3 0.9
Questions to Consider when Assessing Use-associated
Risks

AB
Will the user get appropriate feedback on activation of the desired function/operation?

v is
Calibrati Alarm
Start
on settings

Ad
37
19

Q
0.3 0.9

2 2 Start
Calibrati Alarm
20
on settings

Monitor is
shutting down37
19
©

0.3 0.9
Questions to Consider when Assessing Use-associated
Risks

AB
Will the user get appropriate feedback on successful / failed
completion of the desired function/operation?

v is
?
Ad
Q
2 2
20
©
Questions to Consider when Assessing Use-associated
Risks

AB
• Will the user be able to locate the desired function?
• Will the user understand how to initiate the desired function?

is
• Will the user get appropriate feedback on activation of the

v
desired function?

Ad
• Will the user get appropriate feedback on successful/failed
completion of the desired function?

Q
• What else can the user do wrong when performing the
desired function?
22
20
• What happens if the user performs
– an incomplete operation (or omits it)?
©

– operations in the wrong order?

– operations at the wrong time?
 Known or Foreseeable Hazards and Hazardous
Situations - Examples

AB
Device Hazard* Use error Hazardous Harm***
situation**

is
Glucose monitor Wrong value Device calibration not No alarm at Untreated hypoglycemia

v
performed hypoglycemia

Ad
Software for Incorrect Operator not aware of Measurement Tumour growth not

Q
tumour size measurement the possiblity to expressed in cm3 detected; patient not
measurement change unit settings instead of mm3 given adequate therapy
Injection needle Virus exposure
2 2
Used injection needle
not securely capped
Unprotected
injection needle
Infection (user cut by
needle)
20

* Hazard: Potential source of harm. A hazard may lead to harm if a hazardous situation occurs.
©

** Hazardous situation: Circumstance in which people, property, or the environment are exposed to one or more hazard(s).
*** Harm: Physical injury or damage to the health of people, or damage to property or the environment.
Use Scenarios

AB
Use scenario
Example – ECG Measurement

is
Task 1
1. User inspects ECG cables

v
Ad
Task 2 Expected response
Intended result 2. User checks colour coding scheme
3. User places ECG cables on patient according

Q
Human Interface Device to colour coding

Definition:
22 4. Result: ECG cables correctly placed =
preparation for measurement OK
20
A specific sequence of TASKS
performed by a specific USER
©

in a specific USE ENVIRONMENT

and any resulting response of the MEDICAL
DEVICE

IEC 62366-1:2015 +AMD1:2020

Hazard-Related Use Scenarios

AB
Hazard-related use scenario Example – Set up of MRI scan

is
1. User opens patient ID screen

v
Task 1

Ad
Unexpected response 2. Data from previous patient (name and social
Task 2
security number) are displayed
Hazardous situation

Q
3. User replaces information with that of the
Human Interface Device coming patient
2 2 4. Changes need to be acknowledged
(Do you want to update patient ID? Yes / No)
20

Definition: 5. User is interrupted by phone call

A USE SCENARIO that could lead to
©

6. After a delay of 30 seconds the changes are

a HAZARDOUS SITUATION or HARM omitted
7. User expects patient ID to be updated = MRI
data registered for wrong patient
Hazard-Related Use Scenarios
Normal use Correct use

AB
Use scenario Following good practice
Use error
Task 1

is
A use scenario describes how the
Expected response

v
Task 2 functions of the medical device are
Intended result

Ad
used when the user tries to achieve
a specific result
Human Interface Device

Q
Hazard-related use scenario
Task 1
2 2 A hazard-related use scenario is a
20
Task 2 Unexpected response use scenario where a use error results
Hazardous situation in a hazardous situation
©

Human Interface Device

In identifying HAZARD-RELATED USE SCENARIOS, the MANUFACTURER should investigate not only specific TASKS that the
MANUFACTURER intends the USER to perform, but also other TASKS and actions that the MANUFACTURER does not intend
the USER to perform but are reasonably foreseeable.
Hazard-Related Use Scenarios

AB
• Use scenarios can be described in different ways, e.g. list
Hazard-related use scenario of user tasks or narrative

is
Task 1

v
• Description should include task, sequence of events, type

Ad
Task 2
of user, use environment

Q
Human Interface Device
• Ideally all hazard-related use scenarios (or all use
scenarios) should be evaluated in the summative
2 2
usability testing
20

• If a selection is necessary it should be based primarily on

the severity of harm that could be caused by a use error
©

IEC 62366-1:2015 + AMD1:2020

 Overview

AB
Primary Operating (Hazard-Related) Use Tasks
Functions Scenarios

is
1. Open ‘patient’ screen
1. Enter patient data A. User enters patient data

v
manually 2. Enter patient data (ID, name,

Ad
2. Perform calibration age, weight, height, gender)
B. User imports patient data
3. Perform measurement 3. Acknowledge changes

Q
4. Close ‘patient’ screen
2 2
20
Use Errors
User does not find ‘patient’ screen
©

and doesn’t open the screen Risk Analysis

User enters wrong weight or height
User does not acknowledge changes
Critical Tasks

AB
• Categorisation of user tasks based on the severity of the
potential harm that could result from use errors, as identified

is
in the risk analysis

v
Ad
• Purpose: To identify tasks that, if performed incorrectly or not

Q
performed at all, may cause serious harm (compromised
medical care) = a critical task
22 FDA Guidance 2016
20

• A task in a hazard-related use scenario in which a use error can

lead to significant harm = a critical task
©

IEC 62366-1:2015 + AMD1:2020

Exercise #3: Hazardous situations

AB
• Study the identified risks in Exercise #3 (see the
Dropbox)

is
• For each risk, assess if the hazardous situation is

v
related to a use error, remember (picture 36):

Ad
– Normal / abnormal use
– Correct use / use error

Q
22
• Motivate your assessment by identifying the Use
error and the cause, remember:
20

– Use error = action error (or no action),

caused by either of:
©

• Perception error
• Cognition error
The Usability Engineering Process - Components

1. Use specification

AB
2. User interface characteristics related to
safety and potential use errors

is
3. Known or foreseeable hazards and

v
hazardous situations

Ad
4. Hazard-related use scenarios + selection for
summative testing

Q
5. User interface specification
22 6. User interface evaluation plan
(formative and summative evaluations)
20

7. User interface design incl. formative

evaluation
©

8. Summative evaluation
User Interface Specification

AB
• Identification of testable requirements relevant to the
user interface...

is
• ...including requirements on instructions for use and

v
Ad
mandatory training materials

Q
• Input from use specification, use errors, hazard-related
use scenarios
2 2
• Continuous update during interface development and
20

evaluation
The button shall be at least 30 x 30 mm
©

It shall be possible to open the package with one hand • May be included in e.g. software or system requirements
The injection needle shall be protected with a cap
specifications
The Usability Engineering Process - Components

1. Use specification

AB
2. User interface characteristics related to
safety and potential use errors

is
3. Known or foreseeable hazards and

v
hazardous situations

Ad
4. Hazard-related use scenarios + selection for
summative testing

Q
5. User interface specification
22 6. User interface evaluation plan
(formative and summative evaluations)
20

7. User interface design incl. formative

evaluation
©

8. Summative evaluation
User interface evaluation plan

AB
Shall document:

• Objective for planned evaluations

v is
• Methods for planned evaluations

Ad
Q
For tests:
22
• User groups to be included

• Test environment
20

• Any accompanying documentation to be provided

©

• Any MD specific training to be done before the tests

• Specify formative and summative evaluations further

Formative vs Summative Usability Testing

AB
Formative usability testing
Start Calib- Alarm
ration

is
• Iteratively performed during product development

v
Value
Trend
A Value

Ad
curve
B
• Advisory testing using sketches, prototypes etc.

Q
Low-fidelity prototyping
• Input to interface concepts and design (strengths,
weaknesses, use errors)
22
• No formal acceptance criteria
20

• ”Good usability engineering practice suggests conducting

©

at least one formative evaluation ahead of a summative

High-fidelity prototyping evaluation.” IEC 62366-2:2016
Formative vs Summative Usability Testing

AB
Summative usability testing

is
• Performed at the end of product development

v
Ad
• Validation of the safe use of the user interface

Q
• Formal acceptance criteria
22
20
©
The Usability Engineering Process - Components

1. Use specification

AB
2. User interface characteristics related to
safety and potential use errors

is
3. Known or foreseeable hazards and

v
hazardous situations

Ad
4. Hazard-related use scenarios + selection for
summative testing

Q
5. User interface specification
22 6. User interface evaluation plan
(formative and summative evaluations)
20

7. User interface design incl. formative

evaluation
©

8. Summative evaluation
User Interface Design and Implementation

AB
• Includes formative testing & iterative design and
implementation

is
• Accompanying documentation

v
Ad
• Training materials
(if training is necessary for the safe use of the device)

Q
• Multidisciplinary team
22 Suggested competences (part 1, 5.8): users, engineers,
user-interface specialists, cognitive psychologists,
20
multimedia programmers, usability engineers, marketing
and training personnel
©

Guidance on design and implementation of user interface is

available in IEC 62366-2:2016, chapter 15
Usability Verification

AB
Usability verification is not part of the usability
engineering process in 62366-1:2015 + AMD1:2020.

v is
Ad
Verification is a requirement of the design control requirements of the
quality management system and the requirements of the risk management
process.

Q
Since the user interface is part of the medical device design, these process
2 2
requirements direct the manufacturer to verify the user interface to the user
interface specification.
20

This verification is not part of the usability engineering process

©

requirements of IEC 62366-1:2015 + AMD1:2020.

The Usability Engineering Process - Components

1. Use specification

AB
2. User interface characteristics related to
safety and potential use errors

is
3. Known or foreseeable hazards and

v
hazardous situations

Ad
4. Hazard-related use scenarios + selection for
summative testing

Q
5. User interface specification
22 6. User interface evaluation plan
(formative and summative evaluations)
20

7. User interface design incl. formative

evaluation
©

8. Summative evaluation
The Usability Engineering Process - Components

1. Use specification

AB
2. User interface characteristics related to
safety and potential use errors

is
3. Known or foreseeable hazards and

v
hazardous situations

Ad
4. Hazard-related use scenarios + selection for
summative testing

Q
5. User interface specification
22 6. User interface evaluation plan
(formative and summative evaluations)
20

7. User interface design incl. formative

evaluation
©

8. Summative evaluation
Summative Evaluation of User Interface

AB
Aim: To demonstrate that the device can be used

is
by the intended users, under expected use
conditions, without serious use errors or problems

v
Ad
• Testing of each selected hazard-related use scenario

Q
• Evaluation of Information for Safety
• Data to be analysed to identify potential consequences of
22 all use errors that occurred
• Further improvement needed?
20

• Part of design validation

©

• It is often practical to combine summative usability testing

with validation of market requirements
Summative Evaluation of Information for Safety

AB
• Typically completed prior to

is
evaluation of the remainder of the

v
Ad
user interface.

Q
22
• It is usually a separate usability test
with different users
20
©
Summative Evaluation of Information for Safety

AB
When Information for Safety is used as a risk
control measure, the summative evaluation

is
shall determine that the information

v
Ad
• is perceivable by,

Q
• is understandable to, and
2 2
• supports correct use of the medical device
by
20

users in the context of the intended use

©

environment

IEC 62366-1:2015 + AMD1:2020, clause 4.1.3

Summative Evaluation of Information for Safety -
recommendation

AB
At least:

v is
• Point to information for safety

Ad
• Ask the test person what the

Q
information means
22
• Ask the test person to describe the
20

potential consequence if the

information is disregarded
©

• Document and evaluate the result

Summative Evaluation - documents

AB
• Summative usability evaluation plan
• Test protocols*

v is
• Test results*

Ad
• Summative usability evaluation report

Q
22
20

• Summative usability evaluation data

©

analysis

*not mandatory
Test Settings

AB
• As realistic as possible

is
- actual environment

v
- simulated environment

Ad
- laboratory

Q
2• 2
“Sufficiently realistic to represent actual
conditions of use” FDA
20
©
Test Participants

AB
Who should participate?
• Intended users (doctors, nurses, technicians, general

is
population…)

v
• Unbiased (not employed by the company)

Ad
How many should participate?

Q
• At least 15 from each intended user group – general
recommendation (FDA and EU)
2 2
In which countries should the testing be performed?
20

• EU: No clear recommendation – choice to be justified

• FDA: US residents (exceptions may be considered)
©
Device under test

AB
• Final design (FDA)

v is
Ad
• Not stated (EU) – final design implied by the
process

Q
22
20
©
Performance

AB
• Training = similar to that received by actual users
minimum one hour training decay, day(s) may be required (FDA)

is
• Instructions for use may be available

v
(validated previously or at this occasion)

Ad
• Task-based testing according to protocol

Q
• Observer to document:
- task completion (may include time to complete)
22 - use errors, close calls, other use difficulties
- user comments during testing
20

• Discussion on root causes of use errors, close calls, other use

difficulties
©

• Subjective ratings may be included to study e.g. user satisfaction –

not part of safety evaluation
Close call: Case in which a user almost commits a use error while
performing a task, but recognizes it in time to avoid making the use error
Task Instructions - Examples

AB
Task no Description • Written – Oral
1 Prepare the system for use:

is
a) Press the On/Off switch on the monitor
• Detailed – Non-detailed

v
and follow the instructions on the

Ad
screen.
b) Load reagents into the Reagents rack.
• Questions – No questions
c) Place the Reagents rack into position

Q
in the Blood Analyser Device.

Task no Description
2 2
20
1 Start the monitor
2 To perform this task, please utilize the
user’s manual, chapter 4.
©

Perform a calibration
3 Apply the sensor on the patient’s finger tip
4 Perform a measurement
5 Note the glucose value: ………….
Observer’s Notes - Examples

AB
Task no Requirements for successful completion of task:
4 1. Glucose calibration successfully performed

is
2. Calibration acknowledged

v
Use error occurred: O yes O no Comment:

Ad
Close call occurred: O yes O no Comment:
Other use difficulties: O yes O no Comment:

Q
Successful completion of task: O yes O no Comment:
Participant used the IFU:
Help from moderator required:
O yes O no
2
O yes O no
2 Comment:
Comment:
20
User comments during performance of task:
Additional comments:
©
Acceptance Criteria

AB
• Formal acceptance criteria shall be set

is
• Tasks associated with hazard-related use scenarios, e.g.

v
No use error that may lead to harm shall occur

Ad
No use error that may lead to unacceptable risk of harm shall occur

Q
• Tasks not associated with risk (commercial purposes) –
2 2
“usability goals”– not part of IEC 62366-1
Objective goals, e.g. 80% of the user shall succeed in calibrating
20
the device within ten minutes
©

Subjective goals, e.g. 80% of the users shall rank the easiness of
adjusting the sensor holder as 4 or better on a scale between 1 and 7
Summative Usability Evaluation Plan 1(2)

AB
Information expected for each selected hazard-

is
related use scenario

v
• Evaluation method and rationale

Ad
• Part of interface being evaluated

Q
• Criteria for determining whether the information for
safety is appropriate
2 2• Will training be performed?
20
• Will accompanying documentation be available?
©
Summative Usability Evaluation Plan 2(2)

AB
Information expected for each selected hazard-
related use scenario

is
• How are the characteristics of the test participants

v
representative of the intended user profiles?

Ad
• Justification of the grouping of participants into
distinct user groups for determination of the number

Q
of participants
2 2
• Description of test environment and conditions +
rationale vs the intended use environment
20

• The definition of correct use

©

• Method of data collections for subsequent analysis

of observed use errors and use difficulties
Evaluation

AB
• All use errors, close calls and other use difficulties should be
analysed in detail

v is
• The determination on pass/fail is made subsequent to this

Ad
analysis based on risk management strategies
e.g. No use error that may lead to unacceptable risk of harm shall

Q
occur
22
• Re-design?
20

• Re-evaluation?
©
The Usability Engineering Process - Components

1. Use specification

AB
2. User interface characteristics related to
safety and potential use errors

is
3. Known or foreseeable hazards and

v
hazardous situations

Ad
4. Hazard-related use scenarios + selection for
summative testing

Q
5. User interface specification
22 6. User interface evaluation plan
(formative and summative evaluations)
20

7. User interface design incl. formative

evaluation
©

8. Summative evaluation
 AB
vis
Ad
Q
LEGACY DEVICES - UOUP
2 2
20
©
User Interface of Unknown Provenance (UOUP)

AB
• Definition: User interface or part of user interface of a
medical device previously developed for which

is
adequate records of the usability engineering process

v
of this standard are not available

Ad
• Perform gap analysis

Q
22
• Evaluation may be performed according to Annex C of
IEC 62366-1:2015 + AMD1:2020
20
©
UOUP – Evaluation According to Annex C

AB
• Usability engineering file

is
• Use specification

v
Ad
• Review of post-production information for incidents or
near incidents

Q
22
• Usability-related risk management
20

• New functionality = non-UOUP

©
Tailoring of the Usability Engineering Process

AB
Level of effort and choice of methods may vary based on:

is
• Size and complexity of user interface

v
• Severity of harm associated with use

Ad
• Extent or complexity of use specification

Q
• Extent of device modification to an existing device
2 2
20
• Presence of User Interface of Unknown Provenance (UOUP)
©
Next steps

AB
• Read the standard &
guidance

v is
Ad
• Focus on risks

Q
22
20
• Additional support
https://www.qadvis.com/our-services/
©

www.qadvis.com
©
20
22
Q
Ad
vis
AB