Managing Performance Measurement Data in Health Care

PMHC51 COVER FINAL.
qxp 5/2/2008 9:29 AM Page 1
Managing Performance Measurement Data
Managing Performance Measurement Data in Health Care, Second Edition

in Health Care, Second Edition
Health care organizations often collect data to meet internal needs for quality
improvement, but data are meaningless if they are not interpreted to give
organizations usable information for improving care and demonstrating the
quality of care, treatment, and services provided. Managing Performance
in Health Care
Measurement Data in Health Care, Second Edition, provides an overview of data
management and explains how to use the data management process to improve Second Edition
performance.
This updated book also contains the following special features:

• Tips on data collection and analysis
• Information on using statistical tools such as control charts and run charts to
analyze and interpret data
• Details on incorporating data collection and analysis into everyday activities
to lessen the “burdened” feeling that might come with the data management
process
• Case studies from a variety of organizations, offering lessons learned and
real-world examples, forms, and tools that organizations can adapt for their
own use
About Joint Commission Resources

Joint Commission Resources (JCR) is an expert resource for health care organizations,
providing consulting services, educational services, and publications to assist in improving
quality and safety and to help in meeting the accreditation standards of The Joint
Commission. JCR provides consulting services independently from The Joint Commission
and in a fully confidential manner. Please visit our Web site at http://www.jcrinc.com.
Order Code: PMHC51
Joint Commission Resources (JCR),

an affiliate of The Joint Commission,
is the official publisher and educator
of The Joint Commission.
1515 West 22nd Street, Suite 1300W

Oak Brook, IL 60523-2082 U.S.A.
www.jcrinc.com
00-Front Matter.qxp 5/2/2008 9:31 AM Page i

in Health Care
Second Edition
00-Front Matter.qxp 5/2/2008 9:31 AM Page ii
Executive Editor: Jaclyn Graham

Project Manager: Andrew Bernotas
Associate Director, Production: Johanna Harris
Associate Director, Editorial Development: Diane Bell
Executive Director: Catherine Chopp Hinckley, Ph.D.
Vice President, Learning: Charles Macfarlane, F.A.C.H.E.
Joint Commission/JCR Reviewers: Diane Bell, Mary Kay Bowie, Linda Hanold, Jerod Loeb, Dave Morton, Klaus Nether, Cecily Pew, Stephen Schmaltz,
Sharon Sprenger, Cherie Ulaskas, John Wallin, Debra Zak
Joint Commission Resources Mission

The mission of Joint Commission Resources (JCR) is to continuously improve the safety and quality of health care in the United States and in the international
community through the provision of education, publications, consultation, and evaluation services.
Joint Commission Resources educational programs and publications support, but are separate from, the accreditation activities of The Joint Commission. Attendees at
Joint Commission Resources educational programs and purchasers of Joint Commission Resources publications receive no special consideration or treatment in, or
confidential information about, the accreditation process.
The inclusion of an organization name, product, or service in a Joint Commission publication should not be construed as an endorsement of such organization,
product, or service, nor is failure to include an organization name, product, or service to be construed as disapproval.
© 2008 by the Joint Commission on Accreditation of Healthcare Organizations
Joint Commission Resources, Inc. (JCR), a not-for-profit affiliate of The Joint Commission, has been designated by The Joint Commission to publish publications and
multimedia products. JCR reproduces and distributes these materials under license from The Joint Commission.
All rights reserved. No part of this publication may be reproduced in any form or by any means without written permission from the publisher.
Printed in the U.S.A. 5 4 3 2 1
Requests for permission to make copies of any part of this work should be mailed to
Permissions Editor
Department of Publications
Joint Commission Resources
One Renaissance Boulevard
Oakbrook Terrace, Illinois 60181
permissions@jcrinc.com
ISBN: 978-1-59940-500-1
Library of Congress Control Number: 2008922765
For more information about Joint Commission Resources, please visit http://www.jcrinc.com.
00-Front Matter.qxp 5/2/2008 9:31 AM Page iii
Contents
Introduction ..................................................................................................................................................................vii
Content of the Book ......................................................................................................................................................vii
Acknowledgments ..........................................................................................................................................................viii
Chapter 1: Data Management: An Overview ................................................................................................3

A National Focus on Data Management and Performance Measurement ..........................................................................4
How Data Can Drive Performance....................................................................................................................................6
The Goals of Effective Data Management ........................................................................................................................8
Requirements for Successful Data Management ................................................................................................................9
The Steps to Effective Data Management........................................................................................................................10
Summary ........................................................................................................................................................................16
Chapter 2: Data Collection ................................................................................................................................19

Types of Data ..................................................................................................................................................................19
Sources of Data................................................................................................................................................................20
Planning for Data Collection ..........................................................................................................................................21
Tools for Data Collection ................................................................................................................................................23
Defining Performance Measures to Collect Data ............................................................................................................24
Evaluating Data Quality ..................................................................................................................................................29
Summary ........................................................................................................................................................................31
iii
00-Front Matter.qxp 5/2/2008 9:31 AM Page iv
Chapter 3: Analyzing Data ......................................................................................................................................35

A Brief Introduction to Basic Statistical Terms and Concepts ..............................................................................................35
Tools for Data Analysis ........................................................................................................................................................38
Conducting a Comparison Analysis......................................................................................................................................44
Presenting Analyzed Data ....................................................................................................................................................48
Training in Data Analysis ....................................................................................................................................................51
Summary..............................................................................................................................................................................53
Chapter 4: Using the Data Management Process to Drive Performance Improvement ................57
Prioritizing Areas for Improvement ......................................................................................................................................57
Preparing for Improvement ..................................................................................................................................................61
Defining the Problem ..........................................................................................................................................................66
Redesigning the Process........................................................................................................................................................67
Testing and Implementing the Plan......................................................................................................................................78
Ensuring Ongoing Performance Measurement ....................................................................................................................80
Summary..............................................................................................................................................................................80
Chapter 5: A Data-Driven Accreditation Process ..........................................................................................85

Periodic Performance Review................................................................................................................................................85
Priority Focus Process ..........................................................................................................................................................86
The On-Site Survey..............................................................................................................................................................87
Evidence of Standards Compliance ......................................................................................................................................89
Summary..............................................................................................................................................................................90
Chapter 6: Case Studies—A Spotlight on Success ........................................................................................93

Case Study 1: Novant Health ..............................................................................................................................................94
Case Study 2: Greater Cincinnati Patient Safety ICU Collaborative ....................................................................................97
Case Study 3: Kings Harbor Multicare Center ..................................................................................................................101
Case Study 4: Henry Ford Health System ..........................................................................................................................104
Case Study 5: Holy Family Memorial ................................................................................................................................107
Glossary of Terms..........................................................................................................................................................113
Index ..................................................................................................................................................................................123
iv
00-Front Matter.qxp 5/2/2008 9:32 AM Page v
00-Front Matter.qxp 5/2/2008 9:32 AM Page vi
00-Front Matter.qxp 5/2/2008 9:32 AM Page vii
Introduction
I n many health care organizations, the idea of data manage-

ment has negative connotations. Organization leaders real-
ize they need to collect data, but in many cases have yet to
as, Which processes should we monitor? How do we know
whether a process is performing optimally? If we make
changes to a process, how do we know whether the process
realize the tremendous potential the data management has improved? What, if any, impact have changes to one
process has for improving organization performance. Many process had on related processes?
health care providers—from organization leaders to frontline
staff members—shrink away from activities that involve the Managing Performance Measurement Data in Health Care,
collection and analysis of data. Organizations that foster a Second Edition, helps answer these questions and others, pro-
culture of performance improvement and continuous quali- viding an overview of the data management process and
ty management realize that effective data management is key explaining how the process can be used effectively to improve
to improving performance, enhancing quality, and, ultimate- performance.
ly, saving lives.
Content of the Book
When using data to drive performance, an organization may Chapter 1, “Data Management: An Overview,” provides an
be unsure about where to begin. Most health care organiza- introduction to the data management process and discusses
tions have a quality management or performance improve- the role of data management in improving performance.
ment program in place to improve their care delivery and,
whenever possible, decrease their costs. However, just Chapter 2, “Data Collection,” takes an in-depth look at data
because a program is in place does not mean the organization collection activities and outlines strategies and tips for suc-
knows how to effectively analyze data, present data and dis- cess. Topics covered in this chapter include sources of data,
play findings in a usable format for improvement activities, tools for collecting data, how to define measures, and how to
and find performance trends and patterns that can indicate train staff on data collection techniques.
opportunities for improvement. Questions may arise, such
vii
00-Front Matter.qxp 5/2/2008 9:32 AM Page viii
Sidebar.
A Brief Note About Terminology
This book is intended to help a variety of health care organizations—including hospital, ambulatory care, behavioral health
care, long term care, and home care organizations—address their data management plans. Because each setting has its own
terminology, the text uses the following terms generically throughout:
• Patient—Anyone receiving care, treatment, or services from an organization, including residents (long term care), individu-
als being served (behavioral health care), members (networks and preferred provider organizations), and clients (home care).
• Staff members—Individuals who work within a health care organization, including administrative staff members (such as
receptionists and information technology personnel), clinical and nonclinical care staff members (such as physicians, nurses,
psychologists, anesthesiologists, physical therapists, laboratory technicians, and dietitians), and support staff members (such
as maintenance, housekeeping, and volunteer staff).
• Family—Those who play a significant role in a patient’s life and form a social support system for him or her. (The family
may include people who are not legally related to the patient.)
• Medical record—Clinical record, patient record, health record, chart, health history, and so forth.
• Care—Clinical and nonclinical interventions (care, treatment, or services).
• Health care organization—Any facility that delivers care, treatment, or services, including ambulatory care centers, assisted
living facilities, behavioral health organizations (residential and nonresidential), home care agencies, hospitals, and long
term care institutions.
Setting-specific examples in this book use the appropriate terminology.
See the Glossary of Terms on pages 113–119 for additional performance measurement–related definitions.
Chapter 3, “Analyzing Data,” provides a brief summary of Chapter 6, “Case Studies—A Spotlight on Success,” looks at five
ways organizations can effectively analyze and communicate real-life examples from different health care settings in which data
about data to turn data into information for performance management techniques have been applied effectively.
improvement. Data analysis tools, such as control charts and
run charts, are discussed, as are display tools, including score- No one book can have all the answers, especially on a topic as
cards, report cards, and dashboards. wide ranging as health care data management. However,
Managing Performance Measurement Data in Health Care,
Chapter 4, “Using the Data Management Process to Drive Second Edition, provides a solid foundation on which organi-
Performance Improvement,” discusses how organizations can zations can build, as well as techniques and examples that can
use analyzed data for performance improvement initiatives. It be adapted to all types of settings.
provides explanations of several quality management tools that
use and respond to data, including Six Sigma, failure mode Acknowledgments
and effects analysis, root cause analysis, and plan-do-study-act. Joint Commission Resources wishes to acknowledge the con-
tributions of the various health care organizations and health
Chapter 5, “A Data-Driven Accreditation Process,” looks at systems that shared their outstanding case studies, examples,
The Joint Commission’s accreditation process and discusses and performance measurement tools for this book. These
how data and data management relate to that process. Both the organizations include the following:
presurvey components and the on-site survey are covered. • Henry Ford Health System, Detroit, Michigan
• Holy Family Memorial, Manitowoc, Wisconsin
viii
00-Front Matter.qxp 5/2/2008 9:32 AM Page ix
Introduction
• Kings Harbor Multicare Center, Bronx, New York

• North Shore–Long Island Jewish Health System, Great
Neck, New York
• Novant Health, Inc., Winston-Salem/Charlotte, North
Carolina
• St. Luke’s Hospital, Kansas City, Missouri
• The many hospitals and organizations involved in the
Greater Cincinnati Patient Safety ICU Collaborative,
Cincinnati, Ohio
Thanks are also extended to Kathleen B. Vega for her tireless

efforts in writing and editing this second edition manuscript.
Thanks, too, to Yosef D. Dlugacz, Ph.D., senior vice president
and chief of Clinical Quality, Education & Research, the
Krasnoff Quality Management Institute, a division of North
Shore–Long Island Jewish Health System in Great Neck, New
York, for serving as a content expert for this manuscript.
ix
00-Front Matter.qxp 5/2/2008 9:32 AM Page x
01-Chapter 1.qxp 5/2/2008 9:33 AM Page 1
Data Management: An Overview
Chapter 1
A n organization cannot improve its performance if it does

not first know how it is performing. Likewise, if an
organization has launched an improvement program, it can-
care they provide. As the public demands more concrete
evidence of safety and quality in health care, and as outside
entities—such as quality management organizations and the
not determine the success of that program if it does not federal government—require specific information on which
measure the program’s effectiveness. For instance, a patient to judge health care organizations’ performance, organiza-
safety initiative might not be as effective in improving tions are scrambling to collect data that show how well they
patient safety if the initiative is not accompanied by measure- provide care.
ment activities that show improvement or areas of concern
that need attention. Unfortunately, just because resources are being spent on data
collection doesn’t mean the money is well spent or the data
To determine whether health care quality is appropriate, effi- truly reflect the performance of an organization. In fact, in
cient, cost-effective, and accessible, organizations must measure many cases, organizations work hard to collect data that they
the quality of care they provide. The bottom line is that with- have no clear idea how to use. It is not just the process of data
out measuring, organizations cannot effectively gauge whether collection that is important in assessing performance but
they are performing adequately or exceptionally or whether what an organization does with those data that can make the
their performance could be improved. Whereas some people difference.
think that performance measurement focuses on problems
(mortality rates, infection rates, patient satisfaction rates, and so Quality and performance measurement involve effective data
forth), others see opportunities to provide better care, increase management—collecting, analyzing, reporting, and reacting
customer satisfaction, and gain a competitive edge. to data. Organizations must understand what information to
measure and how to measure it. In addition, organizations
Each year, health care organizations across the United States must know how to interpret data and how to monitor care
spend significant resources trying to measure the quality of through measures on an ongoing basis.
3
A National Focus on Data Management Both IOM reports emphasize that the majority of errors in
and Performance Measurement health care result from faulty systems and processes rather than
Health care organizations are fundamentally obligated to be from the actions of specific individuals, thus highlighting the
accountable for the quality of the clinical and support services fact that the only way to reduce errors is to change systems and
they provide. In recent years, the concept of using data to processes.
reflect quality has gained more attention, leading to calls for
greater transparency and better reporting of performance lev- One of the recommendations given in the IOM reports was for
els. Many government and private entities are working to facil- large companies to use the way they purchase employee health
itate the sharing of performance data among organizations. care to influence the quality and affordability of health care and
These groups are establishing standards of care to educate the to promote advances in health care quality and safety. This rec-
field and the public. Accrediting bodies such as The Joint ommendation provided a basis for the founding of the Leapfrog
Commission require organizations to collect data for compli- Group in late 2000. Consisting of more than 170 companies and
ance, and various national and government entities, such as the organizations that purchase health care for their employees, the
Centers for Medicare & Medicaid Services (CMS) and the group’s aim is to initiate improvements in the safety, quality, and
Agency for Healthcare Research and Quality (AHRQ), associ- affordability of health care by recognizing and rewarding
ate specific data with performance and quality of care. It is no providers who make significant strides in these areas. Because the
longer a question of whether an organization should collect group has significant economic power, it can dictate quality and
data on performance but how those data should be collected safety standards for health care organizations.
and used.
The Leapfrog Group encourages the public reporting of qual-
Defining the Movement ity indicators so purchasing organizations and consumers can
One of the defining moments in the health care safety and make more informed choices about which health care institu-
quality movement was the release of the 1999 Institute of tion they should use. The group ranks hospitals throughout
Medicine (IOM) report, To Err Is Human: Building a Safer the country on promoting a culture of safety, communication
Health System,1 which focused national attention on health among health care workers, communication between doctors
care safety and brought to light the need to measure adverse and patients, success in preventing infections, level of medica-
outcomes and to introduce greater transparency into the tion errors, and level of complications.3
health care system.
The Role of the Federal Government
According to the 1999 IOM report, as many as 98,000 people A major force in health care policy and performance improve-
die each year from medical errors, and more people die annu- ment in the United States is CMS, the government agency
ally in the United States from medical mistakes than highway responsible for administering the Medicare and Medicaid pro-
accidents, breast cancer, or AIDS. The report stressed that grams. This agency has been demanding answers to questions
although not every medical error leads to death, medical errors about quality of care and accountability for the delivery of serv-
can have serious consequences. In 2001, the IOM released a ices for many years. CMS is the predominant economic force for
second report, Crossing the Quality Chasm: A New Health hospitals and physicians, and reimbursement from CMS
System for the 21st Century,2 which suggested restructuring the depends not solely on volume of patients but on the quality of
health care system as a whole, including delivery and payment the care delivered. CMS drives the quality movement by requir-
mechanisms, to provide better patient care and make the best ing organizations to use evidence-based medicine and by paying
use of available resources. Within this report, the IOM recog- for performance (see Sidebar 1-1 on page 5). The organization
nized a “quality chasm,” or gap between the health care that is contracts with medical organizations to ensure the medical care
known to be “good quality” and the actual health care that paid for with Medicare funds is reasonable and necessary, meets
exists in practice. The report addressed several challenges for professionally recognized standards, and is provided economi-
organizations, including the need to incorporate process and cally. Payment is related to good processes and outcomes, and
outcome measures into day-to-day activities. (For more infor- performance is evaluated through objective measures.3 (See
mation on process and outcome measures, see Chapter 2.) Sidebars 1-2 and 1-3 on pages 6 and 7.)
4
Chapter 1: Data Management: An Overview
Sidebar 1-1.
Pay-for-Performance Initiatives
Through collaborative efforts with public agencies and private organizations such as The Joint Commission, the Centers for
Medicare & Medicaid Services (CMS) is developing and implementing several pay-for-performance initiatives to support
quality improvement in the care of Medicare beneficiaries. The following are a few of these initiatives:
• Hospital Quality Alliance (HQA). The HQA is a public-private collaboration to improve the quality of care provided by the
nation’s hospitals by measuring and publicly reporting on that care. The ultimate goal of HQA is to identify a set of quality
measures that would be reported by all hospitals and accepted by all purchasers, oversight and accrediting entities, payers,
and providers. To date, there are 21 measures reported on Hospital Compare, including 10 “starter set” measures, and addi-
tional measures on which hospitals voluntarily report. These measures reflect recommended treatments for heart attack,
heart failure, pneumonia, and surgical care improvement/surgical infection prevention.4
• Premier Hospital Quality Incentive Demonstration. The purpose of the demonstration is to improve the quality of inpatient
care for Medicare beneficiaries by giving financial incentives to almost 300 hospitals for high quality. Under this demon-
stration, CMS collects data on 34 quality measures related to five clinical conditions. Hospitals that score in the top 10%
for a given set of quality measures receive a 2% bonus payment on top of the standard diagnosis-related group (DRG)
payment for the relevant discharges. Those scoring in the next highest 10% receive a 1% bonus.
• Physician Group Practice Demonstration. This demonstration is the first pay-for-performance initiative for physicians
under the Medicare program. The demonstration rewards physicians for improving the quality and efficiency of health
care services delivered to Medicare fee-for-service beneficiaries.
• Medicare Health Care Quality Demonstration. This demonstration is a five-year demonstration program under which
projects enhance quality by improving patient safety, reducing variations in utilization through appropriate use of evidence-
based care and best practice guidelines, encouraging shared decision making, and using culturally and ethnically appropriate
care. Eligible entities include physician groups, integrated health systems, and regional coalitions of the same.5
CMS is not the only government agency helping to shape the consistent implementation of proven life-saving interventions,
quality and performance improvement movement. AHRQ— including creating rapid response teams, using evidence-based
one of a dozen agencies in the Department of Health & care for acute myocardial infarction (AMI), preventing ventila-
Human Services—further supports health care organizations’ tor-associated pneumonia, preventing indwelling venous catheter
efforts to translate research into practice. AHRQ sponsors and infections, preventing surgical site infections, and preventing
conducts research that provides evidence-based information on severe drug events. Many groups—including AHRQ, the
health care outcomes; quality; and cost, use, and access. The American College of Physicians, the American Medical
information that comes from this research helps reduce the risk Association, CMS, the Joint Commission, the National
of harm from health care services and helps health care decision Association for Healthcare Quality, the National Patient Safety
makers—patients, nurses, physicians, health system leaders, Foundation, and the Veterans Health Administration—endorsed
purchasers, and policymakers—make more informed decisions the campaign.
and improve the quality of health care services.6
Participating hospitals collected data on the previously men-
The Push to Save Lives tioned interventions and submitted them to IHI in April and
In addition to the previously mentioned organizations, the May 2006. In June 2006, IHI reported that results had far
Institute for Healthcare Improvement (IHI) has been critical in exceeded expectations. Instead of the original enrollment goal
pushing organizations to use data to drive quality. In December of 2,000 hospitals, the program included more than 3,000.
2004, the IHI launched the 100,000 Lives Campaign, through The hope of preventing 100,000 deaths was also surpassed;
which it worked with other organizations to disseminate data showed that participating hospitals had collectively pre-
improvement tools throughout the United States. The goal of the vented an estimated 122,300 avoidable deaths.7
campaign was to save 100,000 lives by June 2006 through the
5
Sidebar 1-2.
The Nursing Home Quality Initiative
Launched on November 12, 2002, the national Nursing • Percentage of residents who have moderate to severe pain
Home Quality Initiative is an important component of the • Percentage of residents who were physically restrained
Centers for Medicare & Medicaid Services (CMS)’s com- • Percentage of residents who spent most of their time in
prehensive strategy to improve the quality of care provided bed or in a chair
by U.S. nursing homes. The initiative is a four-prong • Percentage of residents whose ability to move about in and
effort, as follows: around their rooms was reduced
1. Regulation and enforcement efforts conducted by state • Percentage of residents with urinary tract infection
survey agencies and CMS • Percentage of residents who have become more depressed
2. Improved consumer information on the quality of care in or anxious
nursing homes • Percentage of high-risk residents who have pressure sores
3. Continual, community-based quality improvement pro- • Percentage of low-risk residents who have pressure sores
grams designed to help nursing homes improve their • Percentage of low-risk residents who lose control of their
quality of care bowels or bladder
4. Collaboration and partnership to leverage knowledge and • Percentage of residents who have/had a catheter inserted
resources and left in their bladder
The initiative involves several chronic care and post–acute Post–acute care
care measures, which are posted on the CMS Web site. These • Percentage of short-stay residents who had moderate to
measures include the following: severe pain
Chronic care • Percentage of short-stay residents with delirium
• Percentage of residents whose need for help with daily • Percentage of short-stay residents with pressure sores8
activities
IHI continues to work on maintaining the gains that have been (For more information on the role of data in the Joint
realized through the six interventions. Its new initiative, called Commission’s accreditation process, see Chapter 5.)
the 5 Million Lives Campaign, asks hospitals to make improve-
ments to protect patients from 5 million incidents of medical The work of the Joint Commission, along with that of each of
harm over a 24-month period, ending December 9, 2008. (IHI the other aforementioned organizations, is critical to improving
estimates that 15 million incidents of medical harm occur in U.S. the quality of health care in the United States and to effecting real
hospitals each year, for an estimated 37 million admissions.) The change. As a group, these organizations are encouraging health
organization is enrolling additional hospitals and also promoting care organizations to respond to the public demand for improved
the adoption of additional interventions.9 safety while providing the public with information to compare
physicians and hospitals. They are improving accountability by
The Role of The Joint Commission objectifying the delivery of services and providing political
The Joint Commission plays a critical role in driving the qual- awareness of patient safety issues. They have launched and are
ity movement. Through its unannounced surveys, along with supporting a movement toward evidence-based care and the use
other components of the accreditation process, the Joint of data to drive performance improvement and increase quality.10
Commission encourages continuous compliance with estab-
lished standards. These standards—particularly as they relate How Data Can Drive Performance
to data collection, patient safety, and performance improve- Virtually every step of the performance improvement process
ment—have broad importance for health care organizations. is grounded in the information that is generated through data
When organizations collect, analyze, and use data to comply management. Without this information, organizations may
with Joint Commission standards, they can develop a detailed draw faulty conclusions, invest resources unwisely, and fail to
picture of patient safety and quality within the organization. maintain improvements.
6
Sidebar 1-3.
The Home Health Quality Initiative
In 2003, the Centers for Medicare & Medicaid Services The quality measures associated with this initiative are a sub-
(CMS) introduced the Home Health Quality Initiative, set of a larger set of OASIS outcome measures that are well
which aims to further improve the quality of care given to known to home health agencies. These measures have been
the millions of Americans who use home health care services. extensively tested and studied and relate to the following
The initiative combines new information for consumers issues:
about the quality of care provided by home health agencies, • Patients who get better at getting dressed
with important resources available to improve the quality of • Patients who get better at bathing
home health care. Like the Nursing Home Quality Initiative, • Patients who stay the same (don’t get worse) at bathing
the Home Health Quality Initiative is a four-prong effort • Patients who get better getting to and from the toilet
that consists of the following: • Patients who get better at walking or moving around
1. Regulation and enforcement activities conducted by state • Patients who get better at getting into and out of bed
survey agencies and CMS • Patients who get better at taking their medicines correctly
2. Improved consumer information on the quality of care (by mouth)
provided by home health agencies • Patients who are confused less often
3. Continual, community-based quality improvement pro- • Patients who have less pain when moving around
grams for home health agencies • Patients who had to be admitted to the hospital
4. Collaboration and partnership to leverage knowledge and • Patients who need urgent, unplanned medical care11
resources
Sidebar 1-4.
What The Joint Commission Requires
Standards in The Joint Commission’s “Improving Organiz- the timeliness of reporting and receipt of critical tests and
ation Performance” (PI), “Leadership” (LD), “Management critical test results.
of Human Resources” (HR), and “Medication Manag-
ement” (MM) accreditation manual chapters all address the The Joint Commission requires organizations to collect data
issue of performance measurement, as does National Patient in the following areas:
Safety Goal 2, Requirement 2C. Through the PI standards, • Staff opinions and needs
organizations are required to collect and analyze data to • Staff perceptions of risk to individuals
monitor performance, make changes in performance based • Suggestions for improving patient safety
on data analysis, and measure to see if those changes led to • Staff willingness to report unanticipated adverse events
improvements and if the improvements were sustained. The • Patient expectations and satisfaction
standards also require new or modified processes to incorpo-
rate testing and analysis to facilitate improvement. The LD In addition, the Joint Commission requires organizations to
standards require organization leaders to allocate adequate measure the performance of several high-risk processes,
resources for measuring, assessing, and improving the orga- including the following:
nization’s performance. The MM standards require an • Medication management
organization to evaluate the performance of its medication • Blood and blood product use
management system, and the HR standards require organi- • Restraint and seclusion use
zations to assess and measure performance related to staffing • Behavior management and treatment
effectiveness. Requirement 2C requires organizations to • Operative and other invasive procedures
measure, assess, and, if appropriate, take action to improve • Resuscitation and its outcomes
7
Data can be used to assess performance and drive improve- Data can also provide organization leadership with the
ment in a variety of ways, including the following: infrastructure necessary to make informed decisions. With
• Evaluating and improving clinical care decreasing reimbursement and a reduction in available
• Evaluating, monitoring the stability, and increasing the effi- resources, organizations need a way to prioritize their opportu-
ciency and effectiveness of organization processes nities for improvement and the resources that will be needed.
• Diagnosing problems and focusing interventions on the Using data and information provides a more objective
most common causes of performance variation approach to setting these priorities.
• Understanding and balancing the relationship among the
organization’s clinical, financial, and operational goals Data can also help organization leaders determine the value of
• Developing and monitoring compliance with evidence-based specific services, such as whether an elaborate and expensive CAT
clinical practice guidelines to improve diagnosis, treatment, scan will result in better patient outcomes. Without data, there is
and individual monitoring no way to assess whether more sophisticated technology should
• Establishing benchmarks for provider performance and be purchased. With appropriate data, leaders can expect to learn
tracking and comparing provider performance over time what the financial and clinical implications of a new piece of
• Empowering staff members to be accountable for improvement equipment are and in what ways the new equipment will be bet-
• Communicating leadership goals throughout an organiza- ter for a patient than a less elaborate piece of equipment.3
tion and supporting strategic planning
• Improving communication between administrative and clin- Health care, as an industry, is rather unique because unlike
ical leadership other business organizations, health care organizations don’t
• Defining and targeting gaps in patient safety and variations generally make good use of quality data and measurements to
from the standard of care that can help identify areas for improve profit margins and enhance performance. It would be
improvement unthinkable if industry giants such as Toyota or Wal-Mart
• Monitoring improvements to determine success or the need ignored data that could reveal problems or opportunities for
for further improvement improvement, yet health care leaders frequently ignore infor-
• Helping organizations determine where to best invest mation revealed in data.3
resources10
The Goals of Effective Data
Decisions based on data, or quantitative information, are more Management
objective than those that are not. When data are available, an To successfully manage data, organizations should set specific
organization is able to further analyze a problem and determine goals. Without goals for the data management process, orga-
the scope and severity of issues. This is also helpful in setting nizations can quickly lose focus. Some specific goals to keep in
priorities for an organization. mind regarding data management include the following:
• Data measures should reflect leadership goals and values.
Teams are most effective when their memberships have been • Measures should be created based on collaboration and
determined based on data about team member performance, communication. When the finance department, quality
team leader competence, and facilitator skills. Data are used to management department, medical leadership, nursing lead-
express the outcomes expected from teamwork and to measure ership, and administration collaborate to develop perform-
team progress. Data about resource needs and justification for ance measures that define clinical variables and shape finan-
the application of resources are critical if teams are to generate cial results, the resulting data will provide a true and clear
a return on the resources consumed. Finally, data are used to picture of what is occurring in an organization.3
ensure that gains made by improvement teams are maintained. • Users of any data should be involved in the process of
collecting, analyzing, and using the data. Not including
When effectively managed, data can help organizations achieve individuals in the data management process for their areas of
accountability for quality care and show the difference responsibility can have a negative effect on how the informa-
between mere compliance with standards and regulations and tion is used.
actual performance improvement.10 The use of data provides • Data should be easy to collect. If staff members need to stop
organizations with a quantitative method for monitoring their their daily activities to collect data, in many cases they won’t,
performance improvement efforts.
8
and data collection efforts can be affected. By incorporating Too many administrators believe that quality management is
performance measures and the data collection process into about certification and regulatory compliance and therefore
daily work life, organizations can collect data with minimal limit the usefulness of the information gathered for those pur-
negative impact on staff members. poses. However, as previously mentioned, regulations, such as
• Data should be simple and understandable. When data are those of CMS, or standards, such as those of the Joint
complex or the intended audience is unsure of the message Commission, provide organizations with high standards of
being sent with data, the data will most likely not be used. care which ensure that proper care is provided and that com-
The presentation of data should also be simple and under- munication is effective.3 (For more about identifying priorities
standable. for performance improvement, see Chapter 4.)
• Data should be concise and precise. Data should be specific
in terms of what is being measured. There should be no Setting the Tone for Learning
questions concerning data accuracy. Leaders set the tone for performance improvement, identifying
• Data should drive decisions, improvements, and actions. This the organizational approach, determining how data are used,
reinforces the need to have a purpose established for gathering and ensuring that performance improvement efforts are com-
data. If the purpose or planning phase of a project is not estab- municated effectively. Leadership must therefore be focused on
lished first, data management efforts will not produce the data management, team development, and the organization-
desired outcomes. These outcomes include the ability to make wide performance improvement process.
data-driven decisions, improvements, and actions.
Leaders who foster a learning environment—a place in which
Requirements for Successful Data all daily activities are viewed as opportunities to learn and
Management expand knowledge—help create a culture where performance
There are several requirements for effective data management, measurement thrives. Leaders who are committed to learning
including leadership support and involvement and a commit- understand that ignoring problems won’t make them go away;
ment of organization resources, such as time, money, equip- it just lets them continue until they cause even bigger prob-
ment, and information. lems. These leaders believe in taking a practical approach to
problem solving, and measurement gives them the tool to do
Leadership Support and Involvement this.
Leadership commitment is critical to establishing a quality cul-
ture and to promoting performance measurement and quality When senior leadership, such as the CEO, endorses and
management throughout every level of care. An organization’s expects 100% compliance with performance measurement,
leaders are responsible for ensuring that data are measured the staff of the organization is more engaged in the process.
effectively, transformed into information, and then used rea- However, senior leadership endorsement of performance
sonably and thoughtfully to achieve improvements. measurement initiatives is not enough to yield a culture of per-
formance measurement. Every physician, nurse, and other staff
Identifying Priorities member must be on board with the performance measurement
One of the most important functions of leadership relative to and data management process.
performance improvement is the identification of priorities for
performance improvement efforts. This is not a capricious Fostering a culture of learning and performance measurement
process. Leaders need to be involved in an organization’s data takes both vision and courage and can initially make managers
management processes to track and review priorities on an and staff members at all levels uncomfortable. Effective leaders
ongoing basis to determine whether any priorities need to be need to challenge the status quo by insisting that their current
revised. systems cannot remain as they are and by presenting ideas for
alternatives that are clearly thought out and articulated. They also
In many organizations, the performance improvement priori- need to recognize that work systems might contain wasteful work
ties tend to be determined by external regulatory requirements. and to insist that this deadwood be found and discarded.
In some cases, administrative leadership and clinical leadership
view regulatory requirements as obstacles to doing their jobs.
9
Sidebar 1-5.
Involving Clinical Leaders
In some cases, physicians and other clinicians may have to be convinced that quality strategies for performance
improvement—such as working in multidisciplinary groups, incorporating evidence-based guidelines into daily practice,
communicating through meetings, performing careful and complete documentation, and analyzing aggregate data for
trends and commonalities, will lead to improved care and a more productive organization.
Physician leadership, in particular, plays a key role in getting other clinicians to support improvement efforts. Physician
leaders can help shape an organization’s measurement strategies and improvement goals, while balancing administration’s view
of cost–benefit trade-offs. Getting clinicians involved in performance improvement can be a challenge. Physicians are often
reluctant to give up time that could be devoted to care delivery in favor of what might be perceived as just more unnecessary
paperwork. They see no reason to change existing processes that seem to work.
To engage physicians, organizations leaders need to highlight the weaknesses in maintaining the status quo. For example, if
an organization is trying to move from a retrospective approach to a proactive one, organization leaders can show the medical
staff exactly how much effort is being spent on quality control rather than on performance improvement. In addition, organi-
zation leaders can show physicians how measurement and improvement help improve overall patient care and potentially save
lives.
Organization Resources The Steps to Effective Data

An organization needs to have adequate resources to perform Management
data management activities. Staffing should be determined Effective data management involves collecting data, analyzing
before data collection begins, and one person or a designated data, and responding to data. This chapter takes a brief look at
group of people should be responsible for overseeing the data these steps, and more detailed exploration can be found in sub-
management process. Other questions organizations will need sequent chapters.
to answer to determine the necessary resources include the fol-
lowing: Collecting Data
• How much time will data collection and analysis take? The first challenge for many health care organizations is to
• Will department heads be responsible for the data manage- identify the data necessary to improve performance. In many
ment activities in their departments? Will the performance organizations, there is a multitude of data, but the data are not
improvement department support the medical staff ’s data always converted into useful information for the intended
management needs? audience. Organizations do not have unlimited budgets for
• What kind of technology is available to support perfor- data collection, so prioritization is critical. Collecting data just
mance improvement and data management activities? to collect data is a waste of critical resources.
• Where will the data be housed?
• What will the budget be for data management activities in Organizations should carefully consider what data need to be
the organization? What factors will be considered in the collected and how they will be used to drive performance.
budget? Some of the different data types include the following:
By knowing the answers to these questions, when it comes • Clinical. These data include patient outcomes and data on
time to make an informed decision, organization leaders know health status, patient functionality, and screening or preven-
where and to whom to go for reliable information. In addition, tion activities. Clinical data may measure a variety of inter-
these questions need to be addressed in order to calculate a ventions, such as surgical interventions; medication therapy;
budget for providing adequate resources to support an effective special and diagnostic procedures; blood usage; rehabilita-
data management and performance improvement program. tion; infection control activities; immunization status; and
(For further discussion on allocating resources for performance breast cancer screening.
improvement, see Chapter 4.)
10
• Perceptions of care or patient satisfaction data. Trends and pat- step in determining the type of data an organization needs to
terns from this type of information are used to identify per- collect is to determine what data will provide the information
formance improvement opportunities. Some of the com- needed. After that is determined, an organization can identify
mon concerns raised in terms of individual satisfaction are whether it is already collecting the data to answer the question,
whether the care provided was correct and appropriate and such as for regulatory or compliance requirements, or whether
whether the environment was pleasant, comfortable, and it needs to create performance measures and processes to col-
conducive to recovery. It is worth noting that individual sat- lect the data.
isfaction questionnaires tend to have a return rate of less
than 35%. Patients who are very pleased or displeased with It is necessary to determine what systems and processes are in
their service are usually the ones who complete surveys. It is place to identify the performance measures needed to evaluate
also extremely important to monitor patient/family com- how effectively these systems are functioning. All organizations
plaints. These complaints can provide insight into potential have developed performance measures, but they may not be
problems with organization processes. This is especially true linked to all the internal systems. Performance measures may
in light of national concerns regarding corporate compliance have been developed in response to regulatory requirements or
issues. to something of interest. If they are not constructed properly,
the value of the data from these measures may be limited.
• Financial data. Financial data can include data on length of
stay, disenrollment rates, charges, and reimbursement prob-
Issues to address in the development of performance measures
lems. In addition, return on investment, margin rates, operat-
are how to identify measures, determine their purpose, define
ing profit rates, profitability, liquidity, and financial activity
them, and test them for validity and reliability. Organizations
measures can be helpful in assessing performance. Many
also need to determine the appropriate sample size, ensure
organizations do not tie financial data into their performance
consistency among individuals in collecting data, periodically
improvement processes, although such information should be
audit data collection activities, and detect trends and patterns
combined with clinical outcome measures. This is especially
of performance that should be evaluated.
true when examining the cost of corrective actions to address
opportunities for improvement and their effectiveness.
To help identify performance measures, organizations may
• Employee satisfaction data. Employee satisfaction data may want to look at what data entities, such as the Joint
include data on absenteeism, employee turnover, grievances Commission and CMS, require organizations to collect. This
and grievance resolution, perceptions of safety, workplace exercise can provide a good foundation for identifying per-
accidents, employee views of management, career opportu- formance measures within an organization. In addition, other
nities, employee perceptions of the work environment and private and government agencies, such as IHI, AHRQ, and the
work load, perceptions of the culture of cooperation and National Quality Forum (NQF), have defined certain meas-
teamwork, and opinions on recognition, benefits, job secu- ures for organizations to collect that assess specific aspects of
rity, and communications. performance. For example, NQF endorses quality and per-
formance measures for national use and promotes the use of
The types of data an organization collects will depend on the evidence-based quality information to develop preferred prac-
information it needs to know. For example, to understand tices for all types of health care settings.
turnaround time in the operating room (OR), data can be
gathered about timeliness of patient preparation, OR readi- There are a variety of ways an organization can create its own
ness, equipment reliability, surgeon start time, readiness of performance measures. One common way is to establish a spe-
appropriate ancillary staff members, availability of required cific numerator that represents the objective of the study and a
documentation, and so forth.3 specific denominator that describes the population of which
the numerator is a subset. By defining measures in this way,
Organizations should identify the problem before talking leaders can objectively and productively study performance,
about the data needed. Often there is a tendency to look for success, and opportunities for improvement.3 For example, if
data before the problem is fully formulated; organizations an organization were trying to measure surgical site infection
should avoid this scenario to make the most out of their data rates, the numerator would be the number of surgical site
collection processes. Once the problem is identified, the first infections during a particular month, and the denominator
11
would be the number of people having surgery within that uals are not familiar enough with the organizationwide perform-
month. By collecting these data and plotting them on a graph, ance improvement program and specific standards to determine
the organization can quickly see the months in which surgical whether there is a need for continued data collection.
site infections were high and how the average infection rate
compares with a national benchmark. (For more information Analyzing Data
about defining performance measures, see Chapter 2.) Data can come from external sources, such as claims data or
patient satisfaction data obtained from an outside vendor, or
As previously mentioned, when defining measures, an organi- internal sources, such as internal survey data, medical record
zation must determine whether a measure is valid and reliable. data, and so forth. Depending on where the data come from,
For a measure to be valid, the data must be reliable. To deter- the analysis process may be different.
mine whether the data are reliable, data elements must be well
defined. An organization should determine whether there are Analyzing External Data
clear and consistent data definitions present for all the data Typically, data obtained from external vendors have already had
elements. It is important that data elements have the same some preliminary analysis done. For example, the data may be
meaning for everyone. adjusted based on severity or risk. If this is the case, it is impor-
tant for the user to understand the approach the vendor has
Many times, data elements are not well defined. This can result used to make its adjustments. Understanding the vendor’s
in data collection that does not provide the information methodology helps answer questions concerning the integrity
desired to evaluate performance. It may then become necessary of the data. It is also helpful if the methodology has been peer
to repeat data collection until the same results are produced for reviewed. This provides the organization with the opportunity
the measure consistently over time and the results truly identi- to evaluate the strengths and weaknesses of the data in measur-
fy the process being studied. Rework is not a positive force for ing and evaluating what they are intended to measure. Some
encouraging personnel to be part of the performance improve- vendors’ methodologies do a better job of measuring outcomes
ment process. than others. The differences in outcomes may be attributed to
different types of diagnosis-related groups (DRGs), risk- and/or
In addition to defining measures, an appropriate sample of the severity-adjustment models, age of the population, and payer.
data to be collected should be determined in order for the data
to be valid. This is done to create a baseline for measurement. Some of the other questions that should be addressed when
Sampling can also be used to evaluate large populations in using external data include the following:
which it is unrealistic to review each item, patient, medical • Were the data from administrative or claims sources?
record, and so on. There are many different sampling tech- • Were the data abstracted from medical records?
niques. Even if an organization is small and there are not many • How many organizations are in the database?
deviations from expected outcomes, there are specific sampling • Do the organizations in the database represent all U.S.
techniques to address such samples. (Sampling techniques are regions for comparative purposes?
addressed in Chapter 2, pages 27–29.) • How are the norms and benchmarks calculated?
• How often is the database updated?
An organization’s data collection needs may vary over time. To
ensure that it is collecting the right data and using those data Analyzing Internal Data
effectively, an organization may want to consider annually evalu- Internal data should be analyzed using statistical and nonsta-
ating its data collection and quality management plan. This tistical tools. These tools can include, but are not limited to,
evaluation can help show what measures are collecting appropri- cause-and-effect diagrams, run charts, control charts, and scat-
ate data and where changes need to be made. One of the most ter diagrams. The type and scope of analysis done will depend
obvious questions to raise during this annual review is whether on the organization and its needs.
people are using the data being collected. If data are not being
used as part of a performance improvement project or to meet To be effective at data analysis, individuals in an organization
the goals of the strategic/business plan, regulatory requirements, need to be trained in the use of statistical tools. Ideally, this
or contractual obligations, then staff members might want to training should include the use of data from the organization.
stop collecting them. This may seem obvious, but most individ- Many training programs include only definitions and concepts
12
concerning the use of performance improvement tools. as the plan-do-study-act (PDSA) process and Six Sigma can be
However, if staff members are able to see their own organiza- used to tie data with specific actions to improve performance.
tion’s data in the training programs, there are more favorable (For more information on PDSA and Six Sigma, see Chapter 4.)
results in terms of using statistical and nonstatistical tools.
An important point to remember is that data must be shared
Specific individuals within an organization should be identi- with the individuals responsible for addressing any issues. In
fied to perform data analysis. Although anyone in an organiza- many organizations, performance data are collected, analyzed,
tion can be assigned to this job, certain disciplines have had and reported to leadership. Even if a problem involves a specif-
more extensive training in the use of statistical tools because of ic unit, many times the data are not reported by or shared with
their background. Laboratory professionals, social workers, that particular unit. Organizations should share data from
and psychologists are just a few examples of professionals who quality control activities, infection control activities, environ-
have been trained in statistical techniques and who can provide ment of care monitoring, and human resources with all appro-
assistance in analyzing data. Another option is to seek the help priate departments. For example, unit managers need to know
of a statistician or biostatistician, if possible. that their equipment is in good working order, and there
should be a mechanism whereby a unit manager receives a
There should be a mechanism in place to periodically evaluate report from individuals responsible for monitoring activities
performance improvement data and its analysis for validity and on his or her unit. When the unit manager is held accountable
reliability. This can be done by having individuals skilled in data for activities on the unit or in the department, that manager
collection and analysis periodically sample data collected by other needs to receive all appropriate data. The type of health care
professionals and evaluate the integrity. In some organizations, organization and its performance measures will determine the
performance improvement professionals conduct these audits. data flow needed to ensure that everyone receives the data
Other organizations have committees that evaluate the data. needed.
Auditing the integrity of performance data on a regular basis can
avoid expensive rework in the data analysis and management One barrier to using data for performance improvement is if
component of a performance improvement program. the data are not complete enough for the user to make a deci-
sion. An example of this would be a group that receives data
To be effective in improving performance, data need to be pre- about length of stay (LOS) without any corresponding utiliza-
sented in a format that is specific to the intended user. If there tion data. For this reason, most organizations have converted
is a large amount of information to report and that informa- from narrative performance reports to dashboards or report
tion is not organized for the user, the data will not be used. An cards, which include multiple measures and allow data results
example of this would be a number of narrative reports con- to be trended over time. These tools also provide an opportu-
taining data that are not trended or do not highlight any trends nity to evaluate the impact of the measures on various process-
or patterns. This disorganization makes it difficult for a deci- es in the organization simultaneously. (For more information
sion maker to understand the issues contained in the reports. about dashboards, see Chapter 3.)
Different tools can be used to present information, such as bar
charts, percentiles, balanced scorecards, and dashboards. (For Performance measures should be trended over time to see
more information on these tools, as well as the data analysis whether there are significant changes, trends, or patterns. The
process, see Chapter 3.) performance of the entire organization and its departments
should be monitored to determine whether an issue is present
Using Data and Information to Drive Improvement throughout the organization or in only one department. There
This final piece of the data management process may be the have been situations in which an organizationwide process has
most critical. As previously mentioned, if organizations do not been completely changed when only one department was
use analyzed data to identify areas for improvement and imple- responsible for the poor performance. If the issues are identi-
ment improvement initiatives, then the data collection and fied and resolved in one department, there may not be a need
analysis process has all been for naught. to change the system for the entire organization. This
approach can result in cost savings and can also assist in
There are a variety of ways organizations can use data to drive evaluating how the organization’s priorities for improvement
improvement. For example, total quality management tools such are communicated to various departments.
13
Sidebar 1-6.
St. Luke’s Hospital
In the early 1990s, St. Luke’s Hospital—a 582-bed, not-for-profit teaching hospital located in Kansas City, Missouri—
embarked on a performance improvement journey to implement the Baldrige model of performance improvement. This
model provides a framework that any organization can use to assess and improve overall performance.
The U.S. Congress established the Malcolm Baldrige National Quality Award (MBNQA) program in 1987 to recog-
nize U.S. organizations for their achievements in quality and business performance and to raise awareness about the
importance of quality and performance excellence as a competitive edge. The president of the United States gives the
MBNQA to a select few businesses, educational institutions, and health care organizations, based on their success in
meeting specific performance-/quality-related criteria. The Baldrige approach represents more than just a quest for an
award, but is a way to manage, benchmark, and demonstrate quality at all levels of an organization.12 Using the award
criteria as a self-assessment tool has helped some organizations understand their system-level operations and educate
their staff members in performance improvement.
Understanding the Criteria

The Baldrige management philosophy is based on the following seven categories:
1. Leadership. This category examines how senior leaders guide and sustain the organization. It also examines the organiza-
tion’s governance and how the organization addresses its ethical, legal, and community responsibilities.
2. Strategic Planning. This category examines how the organization develops strategic objectives and action plans. Also, it
examines how the chosen strategic objectives and action plans are deployed and changed if circumstances require and how
progress is measured.
3. Focus on Patients, Other Customers, and Markets. This category examines how the organization determines requirements,
needs, expectations, and preferences of patients, other customers, and markets. Also, it examines how the organization
builds relationships with patients and other customers and determines the key factors that lead to patient and other cus-
tomer acquisition, satisfaction, loyalty, and retention and to health care service expansion and sustainability.
4. Measurement, Analysis, and Knowledge Management. This category examines how the organization selects, gathers, analyzes,
manages, and improves its data, information, and knowledge assets and how it manages its information technology. It also
examines how the organization reviews and uses reviews to improve its performance.
5. Workforce Focus. This category examines how the organization engages, manages, and develops the workforce to utilize its
full potential in alignment with the organization’s overall mission, strategy, and action plans. It also examines the ability of
the organization to assess workforce capability and capacity needs and to build a workforce environment conducive to
high performance.
6. Process Management. This category examines how the organization determines its core competencies and work systems and
how it designs, manages, and improves its key processes for implementing those work systems to deliver value to patients
and other customers and to achieve organizational success and sustainability. It also examines the organization’s readiness
for emergencies.
7. Results. This category examines the organization’s performance and improvement in all key areas: health care outcomes,
patient- and other customer–focused outcomes, financial and market outcomes, workforce-focused outcomes, process
effectiveness outcomes, and leadership outcomes. Performance levels are examined relative to those of competitors and
other organizations that provide similar health care services.2
A first step in pursuing the Baldrige model was to understand the criteria and how they related to health care. In the early
1990s, the national Baldrige criteria did not yet exist for health care organizations. The state of Missouri was beginning to
(continued)
14
Sidebar 1-6, continued

develop criteria for a state-level award, and St. Luke’s Hospital was asked to help develop that criteria. The leadership from St.
Luke’s also trained to become examiners and judges for the state-level award and spent a great deal of time researching and
studying the criteria. The organization felt that if its leaders became involved in developing criteria and examining other
organizations that this involvement could help its performance improvement efforts.
In 1999, criteria were created for health care organizations on a national level. By this time, the hospital was familiar with
the state-level criteria and could understand how the national Baldrige criteria could be used to manage and improve organi-
zation performance.
When leaders had an understanding of the criteria, they needed to communicate the understanding and meaning of the
criteria to hospital staff members. The organization repeatedly stressed that it was pursuing a new management model, not an
award for its trophy case. Organization leaders didn’t want this to be just another task for the staff, but a shift in thinking
toward complete organization improvement.
Creating a Culture That Supports the Management Philosophy

To help solicit the staff’s participation and buy-in to the management philosophy, St. Luke’s focused on empowering the
workforce and moving toward a team-based approach to performance improvement. Performance improvement teams
became multidisciplinary, and all committee members had an equal say in the performance improvement process. The hospi-
tal also moved toward shared governance in all nursing units. In the past, nurses at the bedside needed to seek input from a
higher-ranking staff member before making a decision related to patient care. With a shared governance approach, the organi-
zation empowered the bedside nurse to have a say in how care was delivered to a patient.
Aligning Organization Performance Measures with Strategic Goals

As St. Luke’s Hospital was in the process of implementing the Baldrige model, the hospital realized it needed to better align
its strategic goals and measurement system. The organization looked at its metrics architecture and realized that it just wasn’t
giving the needed information. By looking at the measurement approaches of other organizations and industries, St. Luke’s
determined that it needed to develop a balanced scorecard.
Developed by Dr. Robert Kaplan and Dr. David Norton, the balanced scorecard approach provides a clear prescription as
to what companies should measure to “balance” the financial perspective with the other perspectives of the organization.13
(For more information on the balanced scorecard, see Chapter 3.) Although not exactly like Kaplan and Norton’s model, the
balanced scorecard at St. Luke’s includes metrics related to the following five perspectives:
1. People. In order to achieve our vision, how do we put the right people with the right skills, doing the right things in the
right places, consistently?
2. Clinical/administrative quality. To satisfy our customers, which processes must we excel at?
3. Customer satisfaction. To achieve our vision, how should we appear to our customers?
4. Growth and development. In order to create sustainability, how must we grow and innovate?
5. Financial. To financially sustain our mission, what must we focus on?
Saint Luke’s Hospital then linked key performance measures to these perspectives. Some of these measures included employee
retention rate, patient safety index, admitting wait times, market share, and total profit margin. In addition, St Luke’s
Hospital adopted these five perspectives as strategic focus areas for the hospital’s strategic plan. Currently, all organizational,
departmental, and individual employee performance goals are developed around these five perspectives.
(continued)
15

In 2003, after many years of work, St. Luke’s Hospital received the MBNQA. It was a long process that did not happen
overnight. By changing the culture, aligning its priorities, and creating a team environment, the hospital was able to achieve
success and realize a management philosophy that guides the entire health care organization to continuous quality and
performance improvement. Since 2003, St. Luke’s Hospital, which is the tertiary hospital for Saint Luke’s Health System,
an 11-hospital health system, is helping to align the key work systems and key work processes throughout the health system
using the Baldrige criteria. In 2006, Saint Luke’s Health System was the recipient of the Missouri Quality Award, a state-level
Baldrige-based award for performance excellence.
Source: St. Luke’s Hospital. Used with permission.
Summary 6. Agency for Healthcare Research and Quality: AHRQ Profile.

By using data wisely, an organization can identify and address http://www.ahrq.gov/about/profile.htm (accessed Feb. 13, 2007).
7. Institute for Healthcare Improvement: Press Release: IHI Announces
variations in patient care and organization performance. Using
That Hospitals Participating in 100,000 Lives Campaign Have Saved an
data wisely involves defining what data are necessary for per-
Estimated 122,300 Lives. Jun. 14, 2006. http://www.ihi.org/
formance improvement, collecting the data, analyzing the NR/rdonlyres/1C51BADE-0F7B-4932-A8C3-0FEFB654D747/0/
data, and responding to the data. By engaging in an effective UPDATED100kLivesCampaignJune14milestonepressrelease.pdf
data management process, an organization can help improve (accessed Dec. 18, 2006).
patient outcomes as well as financial and operational out- 8. Centers for Medicare & Medicaid Services: Nursing Home Quality
comes, ultimately improving organization performance and Initiatives Overview. http://www.cms.hhs.gov/
quality. NursingHomeQualityInits (accessed Nov. 12, 2007).
9. Institute for Healthcare Improvement: Progress in the 5 Million Lives
Campaign. http://www.ihi.org/IHI/Programs/Campaign/
REFERENCES
Campaign.htm?TabId=3 (accessed Dec. 18, 2006).
1. Institute of Medicine: To Err Is Human: Building a Safer Health 10. Dlugacz Y., et al.: The Meaning of Measures: Using Quality
System. Washington, D.C.: National Academy Press, 1999. Management Indicators to Improve Patient Care. North Shore–Long
2. Institute of Medicine: Crossing the Quality Chasm: A New Health Island Jewish Health System, Lake Success, New York: Center for
System for the 21st Century. Washington, D.C.: National Academy Learning and Innovation, 2007.
Press, 2001. 11. Centers for Medicare & Medicaid Services: Home Health Quality
3. Dlugacz Y.: Measuring Health Care: Using Quality Data for Initiatives Overview. http://www.cms.hhs.gov/
Operational, Financial, and Clinical Improvement. San Francisco: HomeHealthQualityInits (accessed Nov. 13, 2007).
Jossey-Bass, 2006. 12. National Institute of Standards and Technology: Frequently Asked
4. Centers for Medicare & Medicaid Services: Hospital Quality Questions About the Malcolm Baldrige National Quality Award.
Alliance. Oct. 24, 2007. http://www.cms.hhs.gov/ http://www.nist.gov/public_affairs/factsheet/baldfaqs.htm (accessed
HospitalQualityInits/15_HospitalQualityAlliance.asp (accessed Feb. Mar. 10, 2006).
6, 2008). 13. Balanced Scorecard Institute: What Is the Balanced Scorecard?
5. Centers for Medicare & Medicaid Services: Press Release: Medicare http://www.balancedscorecard.org/basics/bsc1.html (accessed Aug.
“Pay for Performance” (P4P) Initiatives. Jan. 31, 2005. 1, 2007).
http://www.cms.hhs.gov/apps/media/press/release.asp?Counter=1343
(accessed Aug. 1, 2007).
16
Data Collection
Chapter 2
T he process of data collection is a critical part of the data

management process. Without consistent and well-
defined measures, the resulting data can be flawed. Without
comes. This can be helpful in understanding the profitabili-
ty and efficiency of certain procedures and operations, as well
as in developing a data-driven approach to care. In other
specific and straightforward methods of data collection, the words, primary data can be used to understand the care
process can be a burden and a waste of precious health care delivered in an organization and how to best use resources
resources. Without a well-considered use for data, data can appropriately.1
be collected unnecessarily, just for the sake of collecting data.
Organizations must be careful to refrain from collecting data Secondary data are more administrative in nature and
that are not useful and to prevent or decrease duplicative data include information such as demographics, diagnoses, treat-
collection. ment, medication, laboratory data, and morbidity and
mortality data. Secondary data are typically collected for
Types of Data financial reimbursement and can highlight issues or prob-
There are two main types of data: primary data and secondary lems in an organization that could need attention. Secondary
data. Primary data are more clinically oriented than secondary data do not show the reasons behind any issues or how those
data. Primary data are typically recorded by clinicians, includ- issues may be addressed. Primary data are more suitable for
ing physicians and nurses, as opposed to administrative staff, that task.
such as financial coders. Examples of primary data include dis-
ease-specific data, blood and blood product use data, operative In some cases, secondary data may not be sensitive enough to
and invasive procedure data, and behavior management and accurately represent the quality of clinical care provided by an
treatment data. organization. For example, if a small community hospital does
not have the capability to perform complex cardiac procedures,
When primary data are coupled with evidence-based medi- such as cardiac catheterizations, patients who require those pro-
cine, the resulting information can be used to examine the cedures are transferred to a hospital that is equipped appropri-
cause-and-effect relationship between treatment and out- ately. However, patients who are inappropriate for transfer
19
because they are too ill or are in the end-of-life stage of care administered, a description of any incidents, and so forth.
remain at the small hospital. When secondary data are collected, These reports can be analyzed for a variety of information,
it appears as if the community hospital has a very high mortality including data on patient falls, medications, and so forth.
rate for cardiac patients, and the data imply that the hospital is
• Clinical surveillance. Clinical surveillance involves recording
providing very poor care. In this case, the secondary data do not
specific outcomes of care in a particular patient population
accurately reflect the level of clinical care in the organization.1
(for example, data on the number of myocardial infarctions
that occur in postoperative patients). Surveillance data can
Because of the potential insensitivity of secondary data, many
highlight a patient safety issue and can help an organization
clinicians do not value them and may ignore reports that are
track its performance over time. Surveillance is relatively
generated from them, saying that secondary data are not
easy to do and can be low cost, but it focuses on particular
specific enough to reflect the way care is provided in the orga-
data points and may not highlight global issues.
nization. However, secondary data can be beneficial in identi-
fying potential clinical issues that could need attention and • Observations. In some cases, merely observing staff members
assessing operational and financial efficiency. Secondary data can yield primary data. For example, if measuring compli-
can also serve as benchmarks for comparing performance.1 For ance with recently instituted hand hygiene initiatives, orga-
example, mortality data can help organization leadership mon- nizations can observe staff members over time to determine
itor the delivery of care and the services being offered by an compliance. One caution with observation is that the
organization and indicate a problem that needs further exami- Hawthorne effect may occur: Individuals may change their
nation. Although such data won’t help the organization iden- behavior because they are being watched. For example, staff
tify the cause of the mortality, they can highlight issues that members may or may not be more likely to comply with
need further study. After performance improvement initiatives hand hygiene initiatives when they are being watched.
are put in place, mortality data can help serve as a benchmark
• Patient and/or systems tracers. Within The Joint Commission
to monitor the success of those initiatives.
accreditation process, surveyors “trace” the care provided to
patients from admission to discharge. They also trace the
Sources of Data care provided in certain systems—infection control, data
Primary and secondary data can come from a variety of management, and so forth. Within this tracer process,
sources. For example, there are treatment details in the medical processes and systems are observed, and information is col-
records, nursing data in shift reports, documented incidents in lected that can reveal areas for improvement. Also, organiza-
an adverse event reporting system, patient satisfaction data in tions can conduct their own tracers to help determine areas
surveys, and quality data from indicators collected to satisfy of improvement. (For more information on the tracer
regulatory requirements.2 The following are some of the most methodology, see Chapter 5.)
common data sources:
• Adverse event reporting systems. The Joint Commission requires
• Patient medical records. For each patient served or treated, organizations to define and implement a method for collect-
health care organizations keep a record of events. It is com- ing information on sentinel events, near misses, and other
piled by physicians and other health care professionals and adverse events that occur within the organizations. Although
includes a variety of patient health information, such as organizations must determine their own definitions of sentinel
assessment findings, treatment details, and progress notes. A event, at minimum, the Joint Commission defines it as an
medical record is a valuable source of primary data, includ- unexpected occurrence involving death or serious physical or
ing interventions, screenings, and outcomes. psychological injury or the risk thereof. Serious injury specif-
• Admission and billing records. These sources can provide ically includes loss of limb or function. The phrase or the risk
information on patient demographics, scheduled proce- thereof includes any process variation for which a recurrence
dures, treatments received, and outcomes. would carry a significant chance of a serious adverse outcome.
Such events are called sentinel events because they signal the
• Shift reports. When nurses leave their shifts, they fill out a need for immediate investigation and response. A near miss is
report that describes what occurred on the shift, including an event that does not result in an adverse situation but is one
the status of each patient the nurse served, medications for which a recurrence carries a significant chance of a serious
20
Chapter 2: Data Collection
Sidebar 2-1.
Mortality Reports
Many organizations purchase or review from their corporate offices or clinical associations reports that show the number of
deaths and complications—according to specific diagnosis-related groups (DRGs) or body systems—within a health care
system or region or nationwide. These reports enable organizations to compare their patient care outcomes with those of
similar organizations to identify opportunities for improvement. Many of the reports also illustrate the number and type of
resources that were required to treat patients. This information is helpful in studying trends and patterns of patient care and
the changes seen in the management of patients. The comparative reports and the corresponding information are helpful to
many organizations in establishing clinical pathways. (For more information on clinical pathways, see Chapter 4.)
adverse outcome. Many organizations devise their own Electronic data do have some disadvantages. For example, even
reporting systems to capture sentinel event and near-miss if an organization has access to many electronic data sources,
information. Such a system may involve staff members filling data are often incompatible, and combining data or transfer-
out written reports, sending in e-mail forms, or using a tele- ring data to where they are needed can be challenging. The
phone hotline. Information from that reporting system is then cost involved in customizing software programs to automati-
transferred to a database or spreadsheet for analysis and cally transfer data from one source to another (software inter-
response. Information generated through a reporting system is face programs) can run into thousands of dollars or more.
useful because it provides data on adverse events occurring Many health care organizations have purchased or created soft-
within an organization. These data can be analyzed to show ware packages that do not “talk to one another,” and this
patterns of risk or areas that need improvement. impedes their data sharing. It is important to note that orga-
nizations need to investigate internal data compatibility or
• Complaint data. When patients are unsatisfied with their
transportability before purchasing or creating new software
care, they have an opportunity to share such information
packages. Despite these potential limitations, recent advances
with an organization. These complaint data can be a good
in Internet software and architecture have made it easier to
source of information about areas that need improvement.
share data. The advent of decision-support software is helping
organizations overcome the issues associated with sharing data
Electronic Data Sources
electronically.
In recent years, the computer has played an increasingly
important role in the data management process. Many health
Despite the benefits, using electronic data might not be a
care organizations access data electronically through a variety
viable option for some organizations. For example, an elec-
of sources, including but not limited to the following:
tronic medical record might not be available, and a manual
• Electronic admission and/or billing data
review of paper medical records may be necessary to collect
• Electronic medical records
data. Even if data are collected manually, organizations should
• Electronic medication administration records (MARs)
consider entering the data into an electronic format, such as a
• Computerized provider order entry (CPOE) systems
database or spreadsheet, for analysis. This allows for easier
• Bar-coding systems for medication management
analysis and manipulation.
• Internal databases or spreadsheets
• Laboratory software
Planning for Data Collection
Electronic data have the advantage of being easily assessable Before an organization starts to collect data, it should have a
and reliable. After an organization defines the data to be col- plan. As with any other venture, having a well-considered,
lected, a computer can retrieve the data efficiently, accurately, well-conceived plan can help ensure that the data collection
and consistently. Electronic data can be shared more expedi- process is efficient and effective and that it yields useful, accu-
tiously than manual data and are easier to manipulate for data rate, and reliable data. When developing a data collection plan,
analysis, if needed. organizations should answer the following questions:
21
• What data will be collected? The answers to this question need must know how, when, and why they are collecting data.
to be specific and precise to ensure that the appropriate infor- Staff members need to clearly understand the required data
mation is gathered. For example, organizations that want to elements and their definitions in order to collect accurate
collect data on infection control rates need to identify what data consistently. In addition, staff members need to know
infections they want to monitor and among what group of when to collect data (such as concurrently at admission,
patients. Figuring out what to measure can sometimes seem prior to a procedure, after a procedure, or retrospectively
like the most difficult part of the data management process. after a patient is discharged) and how frequently (daily,
Some organizations might set their sites too low, measuring weekly, monthly, quarterly, and so forth). People using per-
what they have always measured because it is easy and famil- formance measurement data should know how each measure
iar, even though no improvements to the quality of care or is constructed (inclusions and exclusions), why it is impor-
patient safety might be realized. Other organizations might tant to measure, and what medical guidelines, if any, support
go to the opposite extreme and try to measure everything, the measure. Many workers will perform better if they
spreading resources too thin to allow anything useful to be understand the “hows” and “whys” of what they are doing
accomplished in any one area. Another problem arises when and the importance of their work.
organization leaders decide what to measure without getting
Training could take many forms, including in-services,
input from frontline staff members who actually know and
discussions during staff meetings, and return demonstra-
work with the processes. When measures are chosen in a vac-
tions. If an organization has contracted with a software ven-
uum, without the benefit of multidisciplinary input,
dor or performance measurement system, training seminars
improvement projects tend to focus on areas that don’t really
or self-instruction books may be offered to help data collec-
improve performance. There is the added difficulty of getting
tors better understand the specified performance measures
staff members to commit to projects that they are uninterest-
and the need for complete and accurate data collection to
ed in and for which they see no benefit.
obtain valid and reliable information.
• Are we already collecting these data? Organizations collect a
• How do we ensure that data are collected consistently? An
variety of data for regulatory and accreditation bodies. In
organization may need to conduct involved audits that show
some cases, an organization may already be collecting data
whether the process is being carried out consistently. If
that can be useful in answering a particular question, and
audits reveal inconsistency in data collection, further train-
there is no need to reinvent the wheel. In other cases, small
ing may be necessary.
modifications to the measures for collecting regulatory data
can yield the necessary information. • How do we ensure that data are valid? Periodic validation of
data quality should occur to ensure that the collected data
• If not already collected, how will data be collected? An organi-
are accurate and complete. Any inconsistencies should be
zation needs to define both measures to collect data and the
identified and removed. A common knee-jerk reaction is to
processes involved in collecting them. Measures should be
“blame the collector” for invalid data, when what is needed
easy to understand, and the process of data collection should
for improvement are management processes for retraining or
easily fit within daily work flow. As mentioned in Chapter 1,
educating the workers involved, allowing more time for data
it is important to involve relevant stakeholders in the defini-
collection, redesigning data collection tools, and so on.
tion, collection, and analysis of performance measures.
(Ensuring the validity of data is discussed later in this chap-
(How to define measures is discussed later in this chapter, on
ter, on page 29.)
pages 24–29.)
• How do we ensure data security? Data should be readily avail-
• Who will collect the data? This could be an individual, a
able yet protected from unauthorized disclosure or misuse.
group of people, a committee, or a specific job title or func-
Security policies and procedures should address every point
tion within an organization. Data should be collected by
at which data can be accessed. User responsibilities include
whoever has the easiest and most reliable access to them
password protection, physical security of computers, and
within the organization.
protection of hard-copy reports that contain sensitive infor-
• How will we train data collectors? Training data collectors is a mation. Staff members responsible for information systems
critical step in gathering complete and accurate data while need to protect access to computer files and database man-
minimizing variation. Everyone charged with collecting data agement systems, and they need to determine whether each
22
user has access to modify or only view data. An organiza- cial needs. When the hard-copy data are computerized, sharing
tion’s leaders must weigh the balance between security and data with appropriate health care team members is easily
utility. facilitated (as long as the same software for data collection is
available to all those needing the information), especially
• How will we pilot test the data? When a new data process is
through an intranet system.
introduced or new data elements are to be collected, it may
be beneficial to pilot test the process. The new approach can • Records of events. Often, forms are used to gather informa-
be introduced to a limited area of the organization, such as tion and can be modified to collect needed data in a routine
a single department or unit. Initial problem areas can be and reliable way. For example, patient assessment forms can
identified during the pilot test, and the form or method of be used to calculate composite scores or values for the over-
data collection can be enhanced prior to organizationwide all health status of patients. Some performance measures can
implementation. When the pilot version is complete and be derived based on the change in a patient’s assessment
working smoothly, it can be expanded to the rest of the score from admission to discharge. Forms such as operating
organization. room records and birth records also fall into this category.
Forms must be made useful to the organization’s specific
• How will data be shared? Data should be shared throughout
services and populations. For example, a psychiatric hospital
the organization, as appropriate. Resistance to data sharing
used an admission behavior assessment form to begin cap-
can create “data dynasties.” Such resistance may stem from a
turing needed clinical information that it had difficulty col-
perceived power base, attempts to conceal poor perfor-
lecting on a routine basis as part of the medical record. This
mance, or tight departmental budgets. Leaders should
increased the consistency and quality of data needed for per-
encourage data sharing by showing support of and commit-
formance monitoring and improving processes. Plus, it
ment to efficient collection and use of data. One way to do
allowed the hospital to repeat the assessment at various inter-
this is to position a data administrator who reports directly
vals during the treatment period to assess the success of the
to senior management in an upper level of management.
treatment plan.
Tools for Data Collection • Specially designed data collection tools. At times, data collection
Organizations can use a variety of tools to collect data. The fol- may be facilitated by using a specially created tool or form that
lowing are some of them: contains closed-ended questions and a predefined choice of
responses (such as asking a patient to rate his or her level of
• Checklists. A checklist is a common form for gathering data. pain on a scale of 0 to 10, with 0 equal to no pain and 10 equal
An example of a checklist might be a discharge summary to excruciating pain, or asking for the best answer from four
sheet of instructions to be given to a patient upon discharge choices). Then, data can be entered into an electronic file
from the emergency department. Sometimes a checklist will (spreadsheet or database) or other information system for effi-
have blank lines for physicians or nurses to write specific cient and effective data analysis and information sharing.
instructions given to a particular patient and to check off
when the instructions have been delivered. For example, in Surveys
long term care, a checklist is often used to record a resident’s A valuable tool for collecting data is a survey. Surveys can be
skin integrity on a weekly basis. used to capture data on patient satisfaction or patient percep-
• Logbooks. Many departments or units keep logbooks to record tion of care after health care services have been rendered.
such things as patient falls, including the time of the fall, the Survey data can be obtained many different ways, including
environmental conditions, and the outcome of the fall. A nurs- via e-mail, the Internet, personal interviews, telephone conver-
ing home may keep a logbook of residents’ weekly weights or sations, focus groups, or personal interviews or by mailing a
the activities residents have participated in, with additional survey instrument to a patient’s home. Health care organiza-
notations that are pertinent to each. If additional data are tions may coordinate surveys themselves, or they may give a
required for a performance improvement project, a column(s) list of patients to a third party that specializes in these types of
may be added to these logbooks to facilitate ease of data collec- surveys. Similarly, organizations may receive survey responses
tion. Logbooks can be efficiently transformed into electronic directly from patients, or the responses may be sent to a third
files by using spreadsheet or database software packages that are party that performs the data entry and analysis.
readily available and easily customized to an organization’s spe-
23
One important aspect of a survey is the response rate. Ideally, Defining Performance Measures to
a response rate of more than 50% is desired for a valid survey. Collect Data
If the response rate to a survey is significantly less than 50% Using a performance measure or indicator to collect data helps
(such as 20% to 30%), the organization may need to brain- illustrate performance. Using a quality measure or indicator to
storm ways to increase response rates. This may mean shorten- collect data helps indicate quality. Some organizations have trou-
ing the survey tool so it can be filled out more quickly and eas- ble determining what performance or quality measures to use
ily or offering an incentive to those who respond. In addition, and how to appropriately define them. For example, the finan-
those who do not respond need to be examined to study ran- cial manager of a hospice organization was once heard to ask,
domness of nonresponses. “What quality measures can we possibly measure? All our
patients die—our mortality rates would be 100%!” Health care
Often, surveys are sent to a sample of the population of inter- executives and managers are familiar with financial measures—
est to save costs. (Sampling is discussed later in this chapter, on accounts payable, accounts receivable, income, expenses, return
pages 27–29.) However, organizations must ensure that the on investment, and so forth. But as the business world has real-
sample is representative of the group being surveyed. ized, financial measures do not tell the whole story about an
Sometimes the sample size can be adjusted to devote addition- organization. Many organizations use a few key measures to
al resources to developing a better-designed survey tool and assess performance and quality, such as the following:
increasing the response rate. • Key internal process measures
• Key financial measures
When conducting a survey, an organization should be aware of • Innovation and improvement measures
several potential pitfalls, including selection bias, nonresponse • Customer satisfaction and operational measures
bias, sampling error, and measurement error. Selection bias
results from the exclusion of certain groups of subjects so they The hospice financial manager focused on only one aspect of the
have no chance of being selected for the sample, and nonre- organization’s services and may not have understood the opera-
sponse bias results when a large percentage of subjects are tional challenges that exist. Of course, mortality rates would not
unwilling to respond to a survey. Sampling error refers to the be a measure of quality for hospice patients; organizations pro-
differences from one sample to another, even though both sam- viding such services should focus on measures that are important
ples should be equally representative of the population. to their patients’ quality of life, such as process measures regard-
Measurement error refers to errors in the measurement of data. ing patient assessment for safety needs, dietary needs, pain con-
Such errors could include, but are not limited to, the following: trol needs, and appropriate activity levels. One important hospice
• Inaccuracies in recorded responses because of a weakness in outcome measure might be whether the patient is able to stay at
question wording home with adequate supportive care during the hospice period
• Biased response, such as the “halo effect” in a personal inter- and avoid hospitalization. Other important measures could
view where the respondent feels compelled to please the include patient/family satisfaction, the patient’s level of pain con-
interviewer trol, the relief of gastrointestinal symptoms or other physical
• Inadequate or inaccurate responses due to a lack of effort symptoms that interfere with the patient’s comfort, and a per-
made by the respondent ceived level of medical and emotional support.
If constructed, implemented, and analyzed effectively, survey Performance and quality measures should be created with an
data can be a valuable resource for organizations. However, sur- understanding of how data will be analyzed and interpreted, so
vey data that lack credibility and objectivity may be useless infor- the information derived from the data can be used to provide
mation. If decisions are being made based on survey data, a basic better care to patients. Organizations can link quality and cost,
understanding of sampling methodologies, response rates, and meeting the dual goals of delivering high-quality health care
survey errors is necessary to make the correct decisions. and doing so in a cost-effective manner.
24
Sidebar 2-2.
The Consumer Assessment of Healthcare Providers and Systems (CAHPS) Survey
The Consumer Assessment of Healthcare Providers and Systems (CAHPS) Survey is a standardized survey developed by the
Agency for Healthcare Research and Quality (AHRQ) that is designed to assess health care organization quality care informa-
tion from the consumers’ perspective. Through a standardized instrument and data collection methodology, hospitals and
ambulatory care organizations can measure patients’ perspectives on their care. Although many health care organizations col-
lect information on patient satisfaction, there is no national standard for collecting or publicly reporting this information that
would enable valid comparisons to be made across organizations.
The hospital version of CAHPS, called HCAHPS, can be viewed as a core set of questions that can be combined with a
customized set of hospital-specific items. HCAHPS is meant to complement the data hospitals currently collect to support
improvements in internal customer services and quality-related activities. HCAHPS is composed of 18 patient ratings and
patient perspectives on care items that encompass the following seven key topics:
1. Communication with doctors
2. Communication with nurses
3. Responsiveness of hospital staff members
4. Cleanliness and quietness of the hospital environment
5. Pain management
6. Communication about medicines
7. Discharge information
It also includes four screener questions and five demographic items, some of which may be used for adjusting the mix of
patients across hospitals and for analytical purposes. The survey has 27 questions.
In May 2005, the HCAHPS survey was formally endorsed by the National Quality Forum (NQF). NQF endorsement
represents the consensus of many health care providers, consumer groups, professional associations, purchasers, federal agen-
cies, and research and quality organizations. The HCAHPS survey is being implemented nationally as part of the work of the
Hospital Quality Alliance (HQA), a partnership of federal agencies, hospital organizations, consumer and employer groups,
clinicians, and other key national groups interested in quality measurement and public transparency. For more information
on the HCAHPS survey, visit http://www.hcahpsonline.org.3
The NQF also recently endorsed AHRQ’s CAHPS program for ambulatory care, which includes several surveys
that assess the experiences of health care consumers in various ambulatory settings, including health plans, managed
behavioral health care organizations, dental plans, medical groups, physician offices, and clinics. Ambulatory care
survey products include complete survey instruments as well as optional supplemental items that may be added to
specific instruments. For more information on AHRQ’s CAHPS program, go to https://www.cahps.ahrq.gov/content/
products/PROD_AmbCareSurveys.asp?p=102&s=21.
25
The selection of performance measures should be done sys-

tematically. Many aspects of measurement should be consid- Sidebar 2-3.
ered in determining an effective set of measures. The purpose Examples of the Three Types of Measures
of a measure, the dimensions of performance being measured,
and the ultimate strategic goals of the organization must all be 1. Structure Measures
considered when choosing effective measures. • Practitioner licensure
• Staff training
Health care measures can be taken from any point along the 2. Process Measures
patient care time line: at admission or during treatment, after • Screening for symptoms of depression
instructions/education, at discharge, or after the patient is billed. • Appropriateness of psychotropic medications
Data collection may occur while the patient is receiving treat- 3. Outcome Measures
ment or services (concurrent method) or after the patient is • Readmission within 30 days
discharged (retrospective method). Each performance measure • Against-staff-advice discharges
comprises various inputs to the process, the care process itself, • At-staff-request discharges
and the outcome(s) of interest. The outcome of interest may • Reliable change in symptoms (patient rated)
reflect process points or steps in the patient’s plan of care (for • Global assessment of functioning change score
example, whether the patient received a prophylactic antibiotic
preoperatively, how many minutes prior to the surgical incision
it was administered) or a definitive patient outcome (for exam- tor and the denominator. The numerator can represent the
ple, whether the patient developed a surgical site infection). number of events being measured, and the denominator is all
the opportunities when the event could have occurred.
There are three main types of measures: (1) structure measures,
(2) process measures, and (3) outcome measures. A structure Defining the numerator results from asking a specific ques-
measure assesses whether an organization has the resources and tion. For example, if an organization chooses to examine only
arrangements in place to deliver care, such as the number, type, falls that result in hip fractures in diabetic patients over 70
and distribution of health care providers, equipment, and facil- years old, individuals who meet the criteria define the numer-
ities. Another example of a structure measure is voluntary ator of the ratio. If an organization is interested in determin-
turnover of staff. Although less often used, these measures can ing the effect of medication on falls, it might collect data on
be useful in providing data on potential interventions (such as the number of falls that occurred among all patients on seda-
staff overtime). A process measure focuses on one or more steps tives or any other medications.
that lead to a certain outcome, such as whether preventive skin
care was given to patients at risk for developing pressure ulcers. The population from which the organization wants to collect
An outcome measure focuses on the result of the performance information also needs to be determined. If an organization is
or nonperformance of the process, such as which patients who investigating falls that result in hip fractures in the elderly, the
did or did not receive care developed pressure ulcers. By focus- denominator might be the number of patients on a unit who are
ing on the process and outcomes of care, measures can direct over 70 years of age. If the organization is determining the impact
more attention to the needs of the patient and center care of medication on falls, the denominator might be the total num-
more around the patient. (See Sidebar 2-3 at right, for exam- ber of patients in an organization who are on medication.2
ples of the three types of measures.) Sidebar 2-4 on page 27
provides an example of a poorly constructed measure. It is important to use consistent terminology when defining
measures. For example, if an organization is trying to compare
Creating a Ratio its rate of adverse drug events with the rates of other organiza-
Using ratios, percentages, or rates to assess and monitor care is tions in a network or geographic area, the organization must
an effective way to define a performance measure. To define a first be sure that all relevant staff members in the organization
measure in this way, an organization must define the numera- are using the same definition of adverse drug event.
26
Sidebar 2-4.
A Sample of a Poorly Constructed Measure
A poorly constructed measure can yield virtually incomparable results that, when presented, can cause unneeded alarm and
possible apathy toward performance measurement for improvement purposes. An example of this is a home care measure
defined as follows:
Numerator: All home care clients who improved or stayed the same on their recent assessment of activities of daily
living (ADL) compared to their last previous assessment score
Denominator: All home care clients who received an assessment for this time period (that is, current month)
Exclusions: None
This measure’s population (denominator) includes all home care clients who received an assessment for a certain month.
Data from the measure reflect how many of the defined population showed improvement from the last ADL assessment in
addition to how many of the defined population had the same score on the last ADL assessment. This seems straightforward,
until it is known that the assessments occur every 60 days, upon initial admission to home health care, or when a change in a
client’s health status occurs (typically a worsening condition).
Two loopholes make comparisons (both internal and external) for this measure complicated. One is new clients and the
other is clients whose health status worsens. The good news is that an organization in a steady state (that is, with no new
admissions and no clients who suffer a negative change in health status) is in good shape. But because the state of health care
is so volatile, this measure is setting up the organization for possible failure.
The comparison problem lies in the measure’s structure. When no previous assessment exists, the client cannot be put in
the numerator, so new clients penalize the home care organization. Second, when a client worsens and an assessment is con-
ducted (which is certainly a correct action), the comparison from the last assessment will most likely be negative, so again the
client will not be in the numerator. This last “ding” to the home care organization may be appropriate if the organization
could have done something to prevent the condition causing the health status change. If the measure is risk adjusted to the
patient factors, comparisons would be fairer. However, a better measure for comparison purposes might be to exclude new
clients and perhaps clients who undergo emergency assessment due to a status change. Adding the first or both of these two
simple exclusions would allow for an “apples to apples” comparison and more user acceptance for the measure’s results.
Determining Inclusion and Exclusion Criteria tion effort and can be a more efficient way to collect informa-
When defining measures, it is important to consider the pop- tion to improve performance.
ulation under study and to define that population appropriate-
ly. This may include delineating specific groups of patients that With sampling, an organization draws a limited number of mea-
should be included or excluded in the variable. For example, surements from a larger source and then analyzes those measure-
an organization may want to exclude very young patients, ments to estimate characteristics of that population. Statistically
patients on at least four medications, or patients with a change valid sampling methodology is required to obtain valid and reli-
in mental status. It is important to consider those definitions able measurement data and to ensure the credibility of the infor-
up front to ensure the measure is specifically defined and mation gleaned from that data. Sampling error is the unavoidable
reflects the information being sought. potential for error whenever a random sample rather than a
whole population is used, due to the smaller size of the sample.
Using Sampling to Improve Efficiency When using sampling techniques, it is important to reduce the
It is not always possible to collect data on every individual who amount of sampling error as much as possible. Limiting sampling
meets certain criteria. In such situations, some organizations error is linked to selecting the correct sample size for a population
opt to use a sampling technique. This reduces the data collec- and using random sampling to determine the sample group.
27
When Is Sampling a Viable Option? 98%; because of the difference in sample size, a decrease
Each performance measure needs to be examined separately to of 1 case for Hospital B means a compliance rate of
determine whether it is a candidate for sampling. Some per- 50%.4
formance measures tend to have large populations and others
have very small populations. An organization may be able to
partially sample some measures but may need to include all eli-
gible data (100% sample) for other measures. Statistics can
help an organization decide whether sampling is a feasible
option on a measure-by-measure basis.
Determining Sample Sizes

To determine sample size, several factors should be considered,
including the population size, the anticipated rate for the pop-
ulation, and the probability of making an incorrect decision Selecting Representative Samples
about the population. In general, the more variation that exists To ensure that sampled data represent an organization’s mea-
within the population, the larger the sample size needed to sure population, sample cases should be identified using
make correct inferences about the population. Traditionally, acceptable techniques such as the following:
the field of statistics has provided several formulas for calculat- • Simple random sampling. This is a process in which a prede-
ing appropriate sample sizes. Even though these formulas give termined number of cases from a population as a whole are
the most appropriate sample sizes, it may not be feasible for an selected for review. It is predicated on the idea that each case
organization to select the appropriate sample sizes. Limitations in the population has an equal probability of being included
in time and resources may be a hindrance to collecting sample in the sample.
sizes that are defined by statistical formulas. In such cases, the • Systematic random sampling. This is a process in which one
following recommendation may be used: case is selected randomly, and the next cases are selected
according to a fixed interval (for example, every fifth patient
who undergoes a surgical procedure).
Population Size < 50
Collect data on the whole population. Other sampling techniques, including the following, also may be
used, but they can increase the possibility of sampling errors:
• Stratified sampling. In this two-step process, first, the popu-
lation is stratified into groups, and then a simple random
Population Size > 50
sample is taken from each group.
Select a random 10% of the population or select 50
individuals in the population (whichever is greater). • Cluster sampling. In this process, the population is divided
into groups, and then some of the groups are selected to be
sampled. For example, to save costs, an organization may
It must be noted that sample size has an inverse relationship divide a large geographic area into compact regions or clus-
with variation. As the sample size decreases, the amount of ters. The organization can then select a random sample of
variation increases. Consider the following example: clusters for the study.
Hospital A has 50 heart failure (HF) patients being • Judgment sampling. With this method, experts in the subject
sampled to see if they were prescribed aspirin upon dis- matter select certain cases to be sampled. Unlike the previ-
charge. Hospital B has 2 patients in its sample size. Both ously mentioned “probability” sampling techniques, this
hospitals have a compliance rate of 100% for the HF form of sampling is considered “nonprobability” sampling.
Patient Aspirin Prescribed Upon Discharge indicator in It is likely that the sample group will not represent the pop-
the first quarter of 2009. If there is a decrease of 1 case ulation’s characteristics; however, the experts selecting the
in the numerator for this indicator for Hospital A in the cases may be trying to change a particular process. For exam-
second quarter of 2009, the compliance rate will be ple, a study may attempt to determine the reasons women
28
having acute myocardial infarction are less likely to experi- forming correctly or incorrectly. Effective measures possess both
ence thrombosis than men of the same age. high sensitivity and high specificity for a given process.
Implementing Sampling Processes Data will not be collected consistently if the collection process is
A simple method for selecting a systematic random sample overly troublesome or not well defined. Automated sources
would be to obtain a list of patients in the order in which they reduce the burden of data collection and retrieval. User-friendly
were discharged or treated, count the number of patients on the forms and standard measurement tools support efficiency in the
list, and divide by the number needed for the sample size. The collection of accurate, complete data. See Sidebar 2-6, on page
resulting number is the interval between one patient on the list 31, for information on submitting data to a measurement
and the next patient on the list who is to be selected for sampling. system.
The key steps in this process are as follows: Determining the list
(sorted in a way that the cases are randomly ordered, such as by Evaluating Data Quality
discharge date), calculating the sampling interval, and determin- There are many ways to evaluate data quality. Some measures
ing the starting point. For example, if the list has 300 patients, require special validation techniques, depending on their
and the required sample size for the measure is 50 cases, every unique characteristics. How an organization decides to validate
sixth (300 / 50 = 6) patient record would be selected for data its data depends on its operation and needs. The following are
collection. To make sure each patient has an equal chance (ran- suggestions of commonly used techniques:
domness) of being selected, those performing the selection may
roll a die. For example, if the die comes up six, starting at the • Observation. In many cases, observing a random sample of
sixth patient record, every sixth patient record is selected for data staff members and verifying that they are collecting data
collection, until 50 cases have been chosen. If necessary, one can appropriately and at the right time can help verify the data
rotate up to the top of the list to get the required number of cases. collection process. Any time observation is used to collect or
A simple random sample is best obtained using one of many verify the quality of data, organizations should be wary of
available computer programs (often statistical software or data- the Hawthorne effect.
base software) that offer random number generators. • Data collection software. The use of data collection software
allows for built-in edits to promote data integrity. This is a
Ensuring Data Validity and Reliability benefit of using an automated approach to collecting data
As mentioned in Chapter 1, reliable, valid measurement is key because some errors can be found and corrected at the point
to capturing relevant, useful data. Measurement does not have of data entry. Also, missing data can be prevented by soft-
to be burdensome, but it does need to reflect performance in ware prompts to the user for needed data.
an accurate way. Reliability refers to the consistency of mea-
surement, given a specific measurement tool. A measurement • Periodic reabstraction by a different person than the usual data
is reliable if it consistently produces results where the only vari- collector for a sample group of patient records. This approach is
ability is due to the process, not to measurement error. (See commonly used for data that are manually abstracted on a
Sidebar 2-5 on page 30.) A valid measure reflects the process monthly or quarterly basis. A certain number or percentage
or performance it is intended to measure. For example, a mea- of patient records are pulled at random, and someone reab-
sure of depression should reflect the condition of depression stracts the data using the same data collection tool used by
and not anxiety or a sleep disorder. the usual data collector to determine an error rate. This
method is referred to as interrater or interobserver reliability.
In addition to being reliable, measures should be sensitive and (See Sidebar 2-5.) Frequently occurring errors are investigat-
specific. A measure is sensitive if it is able to detect subtle changes ed to determine possible causes, and actions are taken to pre-
in the underlying process or performance—in other words, if it vent the errors from recurring. Some organizations select a
doesn’t produce a false sense that the process is performing cor- statistically valid sample size to reabstract; others simply
rectly. A measure is specific if it measures only the underlying select a small, manageable number of records. The theory in
process or performance and not artifacts or confounders. Simply the latter case is that even evaluating only 10 to 15 records
put, it should not produce a false sense that the process is per- per quarter is better than evaluating none.
29
Sidebar 2-5.
Evaluating Reliability
One way to define reliability is the degree to which multiple Kappa and ICC statistics can range from –1 to +1, with 0
assessments agree. To evaluate whether multiple assessments indicating no agreement, negative values indicating worse
agree, organizations can measure the interrater reliability— agreement than by chance alone (not a likely finding), and
the agreement between reviewers or surveyors. This is done +1 indicating perfect agreement. These statistics are generally
by having all reviewers or surveyors review the same selected considered to indicate fair agreement if greater than 0.4 and
cases. Their answers to each question are then compared excellent agreement if greater than 0.80.
using statistical measures of agreement, and the measures for Consider an example: Two hypothetical physicians evaluate
each question are averaged to produce overall measures of 100 chest x-rays to see if the x-ray findings are consistent
agreement. with pneumonia. To check the interrater reliability, the evalu-
This method of reliability evaluation uses three different, ations of the two physicians are compared:
but related, statistical measures. The proportion of agreement
Physician B
is calculated for all items, and either a kappa statistic (for cat-
Pneumonia No Pneumonia Total
egorical variables) or an intraclass correlations (ICC) statistic
Physician A Pneumonia 75 3 78
(for continuous variables) is calculated.
No Pneumonia 2 20 22
The proportion of agreement is defined as the number of Total 77 23 100
times the raters agree divided by the total number of reviews:
Proportion of agreement = Number of agreements

Proportion of agreement = (75 + 20) / 100 = 95%
Number of possible agreements
Proportion of agreement – Agreement by chance
Kappa =
This is a useful, easy-to-understand measure. For example, 1 – Agreement by chance
if a measurement is made on the same patient 100 times, Agreement by chance = (77 × 78) / 100 + (23 × 22) / 100 = 65.2%
and the same answer is obtained 50 times, the proportion .95 – .652
Kappa = = 86%
of agreement would be 50 / 100, or 50%. Some statisti- 1 – .652
cians believe that a proportion of agreement statistic can
be misleading because there can be some agreement by Both the proportion of agreement (95%) and the kappa sta-
chance, and some ratings will agree even if the ratings were tistic (86%) indicate that agreement between the two review-
randomly distributed. Therefore, it is important to supple- ers is excellent.5
ment this measure with additional statistics.
Kappa and ICC statistics make it possible to measure
agreement among raters beyond what would be expected by
chance. As previously mentioned, a kappa statistic is used to
measure agreement for categorical variables, and a ICC statis-
tic is used for continuous variables. They are both set up as
follows:
Observed proportion of agreement
Kappa and ICC = beyond chance
Number of possible nonchance agreements
30
Sidebar 2-6.
Submitting Data to a Measurement System
An important factor of data quality that directly affects the resulting usefulness of the reports derived from third-party ven-
dors or performance measurement systems is the timely transmission of data to those performing data analysis and creating
the feedback information.
Health care organizations are likely to submit data (billing data, financial data, and so forth) in various formats to
various entities at certain intervals. Data should be exact and should contain accurate depictions of patient care, reflecting
the following elements:
• Correct patient demographic information
• Correct time period of the services provided
• Correct list of services/products provided to the patient
• Transmission to the correct vendor
• Review of vendor reports. If a health care organization elec- most likely the best source of information for checking the
tronically transmits its data to a third-party vendor or per- validity of data used in measures.
formance measurement system, it is likely to receive some
type of transmission report detailing the number of records Summary
received and any potential errors that may need to be cor- Data used to make informed decisions about the delivery of
rected. Reviewing these reports and making timely correc- health care should be of the highest quality possible so that the
tions when needed will help improve data quality. right decisions can be made. Organization leaders need to
• Continuous tracking. It is a good idea to track data quality find- ensure that their decisions are based on valid and reliable infor-
ings on an ongoing basis, along with any improvements used mation. Maintaining data integrity involves effectively deter-
to refine the data collection process. Future staff and managers mining the type of data to be collected, defining measures to
who may be new to the process can use this helpful informa- collect it, and ensuring data quality. Staff members who use
tion. This information can also give credence to the quality of the resulting data, as well as staff members involved in the data
the data when someone (such as managers, leaders, the board collection process, should be involved in determining and
of directors, or surveyors) asks, “How do you know that this designing measures to collect data.
report is accurate and that the data are valid?”
References
• Cross-reference checking of results between similar or comple-
1. Dlugacz Y.: Measuring Health Care: Using Quality Data for
mentary measures. If an organization collects data for similar Operational, Financial, and Clinical Improvement. San Francisco:
measures—for example, for different studies or different reg- Jossey-Bass, 2006.
ulatory agencies—the organization can compare the mea- 2. Dlugacz Y.: The Quality Handbook for Health Care Organizations: A
sures and determine whether the number of cases and the Manager’s Guide to Tools and Programs. San Francisco: Jossey-Bass,
measure rates are the same. If they are not, the organization 2004.
can investigate and try to uncover the causes of the discrep- 3. Centers for Medicare & Medicaid Services: HCAHPS: Patients’
ancies, which are often slightly different data definitions or Perspectives of Care Survey. http://www.cms.hhs.gov/
HospitalQualityInits/30_HospitalHCAHPS.asp (accessed Aug. 1,
measure exclusion requirements between the two entities.
2007).
• Networking with other health care organizations that use the 4. Hussain R.: Central Tendency. In Dlugacz Y.: The Meaning of
same or similar performance measures. It is a good idea to ask Measures. Great Neck, NY: The Krasnoff Quality Management
Institute, 2007.
other organizations how they validate their data and to share
5. Eichorn E.: Reliability. In Dlugacz Y.: The Meaning of Measures. Great
lessons learned. Also, measurement systems do their own
Neck, NY: The Krasnoff Quality Management Institute, 2007.
data quality checks and audits of organization data. They are
31
Analyzing Data
Chapter 3
T he mere mention of data analysis often has people say-

ing, “I can’t do that.” However, data analysis and report
development need not be complicated, and they are key
other organizations, to benchmarks, or to target performance
levels. Data analysis can be broken down into two basic steps:
(1) initial exploratory analysis and (2) further analysis. A fun-
components of data management. Rather than rely on intu- damental knowledge of basic statistics is needed to begin
ition or hunches, proper data analysis allows for data-driven analyzing data, including understanding the type of measure-
decision making. ment data collected, the underlying data distributions, the
use of the measures of central tendency, and how variation is
Raw data cannot be used to draw conclusions about a process measured by the range or standard deviation of the data. The
or an outcome. Harvested from medical records, patient sur- following sections provide a brief introduction to statistical
vey results, or some other source, raw data need further pro- analysis.
cessing and refinement to become useful information to
those who manage and deliver patient care. Attributes Data Versus Continuous Data
For the sake of analysis, data can take two main forms: (1)
Some basic statistical knowledge and even creativity are attributes or (2) continuous. Attributes data are also referred
required for successful analysis, display, and use of perfor- to as “binary,” “discrete,” “categorical,” or “count data”
mance measurement data for the purposes of understanding, because of their finite nature. Within such data, the number
forming conclusions, and/or asking more questions about of individuals who experienced the attribute of interest are
underlying performance. counted from all those who had the potential to experience
the event of interest. Because the results have limited possi-
A Brief Introduction to Basic bilities or categories, the range is limited to the number or
Statistical Terms and Concepts percentage in each category. Attributes data include yes/no
An organization’s performance can be evaluated both using types of measures, such as patient had a Cesarean section (C-
internal comparisons over time and external comparisons to section) delivery/patient did not have a C-section delivery;
patient lived/patient expired; and patient acquired a pressure
35
ulcer/patient did not acquire a pressure ulcer. Attributes data cols are working as designed. Suppose that the organization’s
can be evaluated using a bar chart or frequency table. (See number of patient falls averages 20 per month and ranges
pages 38–44 for a discussion of data analysis tools.) between 17 and 23 per month in a single year. This would sug-
gest a stable process because the variation is predictable within
Continuous data, also called “variables data,” include informa- given limits. This type of variation is said to be due to common
tion such as length of stay, time to first dose of an antibiotic, causes.
and cost of services. These data can have an indefinite number
of potential outcomes (such as 2 minutes, 35 minutes, 359 When an observed variation is due only to common causes, but
minutes, and so forth). Evaluation of the data includes exam- the rate being measured—in this case, the average fall rate—is
ining the variability of the results or outcomes. A histogram is unacceptable, it would be appropriate to try to improve the
a form of exploratory analysis typically used to examine and process by introducing a new or modified process. For example,
display continuous data. A histogram is not typically used for the organization might implement a fall prevention program that
attributes data where only two possible outcomes exist. incorporates interventions related to employees, equipment,
environment, supplies, and/or policies/procedures.
Common Cause Variation and Special Cause Variation
Variation exists in all human endeavors, but whether the varia- Now suppose that, during the year, the organization sees the
tion is due to common causes or a special cause indicates a par- average number of falls remain the same, but in one month
ticular response. Common cause variation is best described as there are 35 falls. This change in variation would be due to a
“noise,” or random variation, which can result from many factors special cause. In this case, the special cause would be a negative
inherent to a process. It is produced by the interaction among or an undesirable finding. The organization should not make
variables within a process. For example, if you write the letter B any change in its fall prevention protocols until the special
four times with your dominant hand, the letter will look slightly cause is identified and eliminated.
different each time. This variation in appearance is due to a vari-
ety of factors—hand pressure, hand angle, amount of ink used, Now suppose that the number of falls in the second year
and so forth. These factors are common cause variations. decreases to an average of 17 per month, with a range of 14 to
19, after the introduction of a prevention program. This
Common cause variation does not imply that a process is func- change would be a special cause that is positive or desirable.
tioning at either a desirable or an undesirable level; it only
describes the nature of variation—that it is stable and pre- Neither common cause nor special cause variation is good or
dictable within given limits or that it is in statistical control. In bad in itself. For example, if one does not plan for special cause
the handwriting example, common cause variation looks at the variation, then such variation is usually undesirable. However,
slight differences between the letter, not whether the handwrit- when an organization makes a change to improve a process, a
ing is legible. special cause can signal that the change effort has been effec-
tive. On the other hand, a process that exhibits a common
Special cause variation is variation due to an unpredictable cause variation may be stable and predictable but totally unac-
event or factor—one that is not inherent to the process. For ceptable. For example, a heart surgery mortality rate of 10% is
example, if you wrote the letter B three times with your dom- stable but totally unacceptable.1
inant hand and once with your other hand, the B that was
written using the nondominant hand would have variation Tests for Special Cause Variation
due to a special cause (you switched hands). With the presence There are three common tests for identifying special cause
of special cause variation, a process is considered to be out of variation, as follows:
statistical control, unstable, and not predictable within limits. 1. A single data point beyond the upper or lower control limits
It does not necessarily mean that the process needs to be fixed; 2. A run of eight consecutive data points on one side of the
it just means that something unusual has happened. mean
3. A trend of six consecutive data points steadily increasing or
Consider the following example: An organization decides to decreasing
monitor fall rates to determine whether its fall prevention proto-
36
Chapter 3: Analyzing Data
Control charts are useful tools for detecting special cause vari- When data are plotted on a graph, the resulting line tells a
ation for performance improvement purposes—both to inves- great deal about the data. If there is no particular lopsidedness
tigate problematic time periods and to affirm whether a in the data, the line will look like a bell curve. A bell curve, also
planned improvement is having the desired effect. (See pages called normal distribution, is symmetrical. It is high in the mid-
42–44 for more information about control charts.) dle and trails off evenly to both sides. When data analysis
yields a bell curve, the center is the mean, or average, of the
Investigating Special Cause Variation data set. The median and the mode are located at the same
Determining whether a special cause is a positive thing to be kept point as the mean. Extremes are rare and fall at the ends of
or a negative thing to be eliminated depends on the context of either side of the bell curve.3 Many health care data are nor-
the measure. Even if a special cause occurred some time ago, it mally distributed, including blood sugar levels, body tempera-
may be worth investigating to determine what happened at that tures, and height of the population.
point in time so the cause can be prevented from recurring and
affecting future performance. If a measure has special cause vari- For normally distributed data, 68.2% of the data would be
ation or if an organization is simply interested in improving its expected to fall between +1 and –1 standard deviation (SD)
overall performance by eliminating common cause variation, an from the mean, 95.4% would fall between +2 and –2 SD, and
organization can use one of several performance improvement 99.8% would fall between +3 and –3 SD. If the plotted data
tools, such as brainstorming, multivoting, and cause-and-effect do not make a bell-shaped curve, the data may be too few in
diagrams. (For more information on these tools, see Chapter 4.) number (that is, small n), or they may be skewed to one side.
Measures of Central Tendency Having an adequate sample size is imperative in assessing the
Central tendency is the middle, midpoint, or typical value of true underlying distribution for any data set. When measure-
data. When data are being analyzed or explored in the aggregate, ment data have a small number of observations (n), the data
it is important to measure the central tendency. Identifying cen- tend not to show a normal shape. Hence, the previously
tral tendency can be done using three calculations for continu- described measures of central tendency will not likely be equal.
ous variables and one calculation for rate variables. The degree to which they differ can be related to a small n, pre-
cluding meaningful statistical evaluation, and/or data that are
The following are the central tendency calculations for contin- skewed to the left or right. While the mean is the most com-
uous variables: monly used measure of central tendency, the median is often
• Mean. This is the simple mathematical average. It is calculat- used instead of the mean when data have a small n because the
ed by adding all the values for a particular variable (in the mean can be skewed by outliers in a small sample size, and the
sample) and then dividing that number by the sample size. median is less sensitive to outliers in this situation.
• Median. This is the middle value of the variable. To find this
calculation, all the values for a particular variable (in the Variation and Standard Deviation
sample) are ranked from least to highest. If the sample size is Central tendency alone is not a good way of describing a sam-
an odd number, the middle value in the series is the median. ple or population. Along with this aggregate number, it is crit-
If the sample size is an even number, the two closest values ical to describe the variation. The variation can be thought of
in the middle of the series are averaged. as the dispersion or spread in the data. In other words, it is a
• Mode. This is simply the most frequently occurring value for measure of how individual values cluster around the central
a particular variable (in the sample). tendency. An increase in variation indicates that the data are
further away from the central tendency.
The following is the central tendency calculation for rate
variables: There are many calculations for variations. One of the most
• Overall rate. This is the sum of all numerators (that is, readily used methods is standard deviation. Standard deviation
events) divided by the sum of all denominators.2 is a measure of variability that indicates the dispersion or
spread in a distribution. It is a more exact measure of variabil-
The mean, median, and mode are commonly used measures of ity than the range. Whereas the range is the difference between
central tendency in exploratory data analysis. the largest and the smallest observations, the standard deviation
is the amount of variability or spread around the mean. It is
37
equal to the square root of the variance, which measures the data over time. Stock market data are commonly presented in
average squared distance from the mean. line charts to show valuation trends over time for a particular
stock. Some fluctuation is expected, but any peaks or valleys in
The following is the mathematic calculation for standard the graph are studied for causative factors.
deviation4:
√
∑ (xi – mean)2
n–1
Tables
A table or a spreadsheet, which can be computer generated, can
help gather a great deal of information in one place and is use-
The statistical formula for calculating the standard deviation ful when comparing data of different types. Tables are also
for continuous data looks complex but is fairly easy to under- helpful for analyzing multiple factors simultaneously.3 For
stand when it is broken down. (See Figures 3-1 and 3-2 on example, an organization looking at fall rates can use a table to
pages 39 and 40.) examine not only the rate of falls but the demographics of the
individuals falling and the times of day they fell.
Tools for Data Analysis
Organizations can use several tools to help with data analysis. Pie Charts
Depending on the nature of the data, different tools may be more A pie chart (see Figure 3-5 on page 41) can show how various
appropriate than others. Some tools are used to initially analyze factors fit into a larger picture. It helps organizations view data
the data, and others are more appropriate when determining how at a glance and see a smaller piece and its relationship to the
to respond to the data. The following sections discuss some of the whole. For example, if an organization is looking at the desti-
most common tools used in initial data analysis. (Tools used to nations of patients after discharge, a pie chart could show what
design performance improvement initiatives are discussed in percentage of people were released to home or self-care, a
Chapter 4.) skilled nursing facility, a rehabilitation center, a hospital, and
so forth.
Bar Charts
A single measurement value has little meaning unless it can be Histograms
compared to some other referenced number for evaluation or A histogram (see Figure 3-6 on page 42) is a vertical bar graph
evaluated with successive measurement values to study per- that is used to determine the shape of a data set—its disper-
formance patterns and trends over time. Comparison data are sion, central tendency, and overall distribution. A histogram
not always readily available, and if an organization’s results are shows the range of variation of observed measurement data by
unstable from one time period to another, comparisons may be determining the spread of the data and which outcomes are
meaningless. In evaluating measurement changes over time, most common. Its main use is to determine whether the data
organizations can use simple graphs and charts to look for are normally distributed, skewed to the left or right, or
trends and patterns that indicate unstable performance. bimodal (with two “humps,” or bell-shaped curves).
Bar charts (see Figure 3-3 on page 40) are helpful when data Scatter Diagrams
consist of distinct or different categories. For example, an Scatter diagrams are graphic representations of data that depict
organization evaluating the number of individuals served in a the possible relationship between two variables. They show
group home who suffered infections as categorized or grouped what happens to one variable when another variable changes to
into ear infections, conjunctivitis, and urinary tract infections, test a theory that the two variables are related. A scatter dia-
may want to use a bar chart. The x-axis defines one variable gram cannot prove that one variable causes the other, but it
(the number of individuals) and the y-axis defines another (the does indicate whether a relationship exists and the strength of
different types of infections). The variability of the results or that relationship (positive, negative, or zero).
outcomes can be estimated using the number of observations
to calculate a standard deviation. Run Charts
A run chart (see Figure 3-7 on page 43) plots data points in a
Line Graphs time sequence graphically to show whether patterns or trends
For time-ordered data, bar charts may not be the best choice. can be attributed to common or special causes of variation. A
Line graphs (see Figure 3-4 on page 41) are used to evaluate run chart can be a good starting point for analyzing data
38
Figure 3-1. Calculating Standard Deviation for Continuous Data
This step-by-step approach makes the formula for calculating standard deviation easier to understand and use.
√∑
∑
√
39
Figure 3-2. Calculating Standard Deviation

This example illustrates how to calculate standard deviation of length of stay (LOS) (days) for two units in a hospital. It
should be noted that length of stay is a continuous variable. Each unit has only three patients.
Unit 1 Unit 2
LOS (days) LOS (days)
Patient X 5 Patient A 5
Patient Y 1 Patient B 6
Patient Z 9 Patient C 4
Mean 5 Mean 5
5–5=0 Then (0)2 = 0 5–5=0 Then (0)2 = 0
1 – 5 = –4 Then (–4)2 = 16 6–5=1 Then (1)2 = 1
9–5=4 Then (4)2 = 16 4 – 5 = –1 Then (1)2 = 1

∑ = 32 ∑ =2
Then √ (32 / 2) = 4 Then √ (2 / 2) = 1
The standard deviation for Unit 1 is 4, and the standard deviation for Unit 2 is 1. Even though both units have an aver-
age LOS of 5 days, Unit 2 has less variation. If all characteristics are the same and patients are at the same level of severity,
it can be argued that Unit 2 patients (on average) will have a higher probability of leaving after 5 days than patients on
Unit 1. This is because Unit 1 has so much more variation.4
Figure 3-3. A Bar Chart
This figure shows a typical bar chart. Bar charts are helpful when evaluating data that fall into distinct categories or time periods.
Number of Patients
40
Figure 3-4. Line Graph
Line Graph of Referral Call Data: Monthly referral calls received. Line graphs are particularly useful for illustrating data over time; this
graph shows the trend toward an increase in monthly referral calls. This may also serve as the basis for a run chart. (See Figure 3-6.)
Figure 3-5. A Pie Chart
Pie charts are used to visually illustrate a data point’s relationship to the whole.
41
Figure 3-6. Sample Histogram

This histogram shows the distribution of times to administration of thombolytic therapy for acute myocardial infarction (AMI) patients
at one hospital. Because the goal is to administer therapy within 30 minutes, a nonnormal distribution would be expected (skewed to
the left).
quickly as it is very simple to construct and interpret. It can be A shortcoming of run charts is that they are insensitive to
used with any type of data, including measurement data, count extreme values that lie far above or below the center line, and
data, percentages, and ratios. Interpreting a run chart involves errors in interpretation can occur because a special cause
counting the number of runs and looking for trends in the variation is overlooked. Distinguishing common cause from
data to identify any special causes of variation. A run is defined special cause variation in a run chart requires thoughtful eval-
as one or more consecutive data points occurring on the same uation of measurement values that are extremely different from
side of the center line. (The mean is used as the center line in other data points.
a run chart.) A single point on a line is not counted; multiple
points occurring on the line are assigned alternately above and Control Charts
below the center line. A trend is an unusually long series of Control charts are used to assess the stability of a particular
consecutive increases or decreases in the data. Typically, six or process through statistical analysis of variation in a measure’s
seven consecutive data points show a trend.1 There are three performance over time. After a norm is established, control
tests for interpreting run charts: charts can concurrently monitor whether there is excessive
1. The presence of too much or too little variability. In most deviation from that norm or standard of care. By graphing
cases, it is best to have at least 16 points in a run chart, variability from a predetermined standard, an organization can
excluding those on the mean, to effectively identify a spe- monitor when a process is out of control and can take steps to
cial cause. After 25 data points, additional data do not address the problem.5 (See Sidebar 3-1 on page 44 for more
appreciably increase the power of tests. Depending on the information.)
number of data points in a run chart, a minimum number
of runs indicates common cause variation. For example, if Variation is expected in performance measurement data, but a
a run chart has 24 data points, it should have at least 8 control chart can identify random versus nonrandom variation.
runs. Fewer than 8 runs signifies that one or more special Control charts indicate whether a process is “in statistical con-
causes are present. trol” (that is, stable with only common causes of variation) or
2. The presence of a shift in the process. A special cause exists if “out of statistical control” (that is, unstable because of variation
a run contains too many data points. due to special causes). Being in statistical control does not neces-
3. The presence of a trend. A special cause exists if a run shows sarily mean that performance is acceptable. It is possible to have
a trend.4 stable and predictable performance that is substandard.
42
Figure 3-7. Run Chart
Run Chart Interpretation: Monthly referral calls received (18 data points). This figure shows the line graph from Figure 3-4 turned
into a run chart by drawing in the median line.
43
Sidebar 3-1.
A Background on Control Charts
Control charts are also called “Shewhart charts,” after Walter Shewhart, who first developed statistical process control
(SPC) for manufacturing in the 1920s with the goals of meeting or exceeding customer expectations, decreasing process
variation, and increasing productivity. Shewhart is also the originator of the plan-do-check-act (PDCA) process improve-
ment cycle, which was designed to work in conjunction with control charts. (The PDCA process is discussed in further
detail in Chapter 4.)
Various statistical tests for special causes can be found in statis- on page 41 provides a case example that includes control chart
tical process control textbooks. The Joint Commission cur- usage.
rently uses the following three common tests to determine
whether a process is out of statistical control: P-charts and u-charts can be viewed individually, whereas the X-
1. The average outside the control limits bar and S charts should be viewed together. To correctly interpret
2. A run of eight consecutive data points on one side of the X-bar and S charts, you must first interpret the S chart. If the S
center line (overall process mean for the control chart) chart is in control, you then interpret the X-bar chart. If the S
3. A trend of six consecutive data points steadily increasing or chart is out of control, there is no point in trying to interpret the
decreasing X-bar chart because it would be invalid. Instead, the special cause
in the S chart should be investigated and removed.
Measures should have a minimum of 12 to 15 data points
before control charts can begin to be interpreted; the upper The XmR chart (also called the “individual’s chart”) is the con-
and lower control limits are considered trial limits until 20 to trol chart of choice for analyzing a single measurement. As
24 data points are present. with the X-bar and S charts, the XmR chart is also a paired
chart. The XmR chart is one of the easiest control charts to
A control chart consists of two parts, as follows: construct and also potentially is less sensitive to finding special
1. A series of measurements plotted in time order causes than the more traditional control charts. This is because
2. A “template” that consists of three horizontal lines called the XmR chart does not use standard deviation to calculate the
(1) the center line, (2) the upper control limit (UCL), and control limit lines as do the other charts. It is mathematically
(3) the lower control limit (LCL)6 impossible to calculate the standard deviation of only one
observation. Instead, an XmR chart uses the average moving
The UCL and the LCL are calculated by multiplying the stan- range of the process to calculate natural process limits.
dard deviation by three and adding it to the mean for the UCL
and subtracting it from the mean for the LCL. Conducting a Comparison Analysis
After conducting an initial analysis of data, data for perfor-
Choosing the Right Control Chart mance measures need to be analyzed for their stability and
There are many different control chart types. Selecting the level of performance. Whereas staff members can use run
correct control chart for the type of data collected makes inter- charts and control charts to evaluate whether an organization’s
pretation more sensitive for detecting special cause variation. performance is stable or unstable, they should use comparison
The four common types of control charts are as follows: analysis to evaluate the level of performance.
1. P-charts, which display proportions
2. U-charts, which display ratios Comparison analysis is a necessary adjunct to run charts and
3. X-bar and S charts, which display continuous measures control charts because it tells whether an organization’s level of
4. XmR charts, which display a small sample of continuous data performance is similar to or different from that of other orga-
nizations and because it is possible to provide a substandard
Table 3-1 and Figure 3-8, on pages 45 and 46, describe and level of care consistently (that is, stable process on a control
show examples of the four types of control charts. Sidebar 3-2, chart but poor comparative performance).
44
Table 3-1. Common Control Charts

Control Chart Type Calculation for the Performance Measure Measure Examples
p-chart n-bar × p-bar > 5 Acute myocardial infarction (AMI) mortality:
Numerator: AMI patients discharged with a status of “expired”
Denominator: All AMI patient discharges
u-chart n-bar × u-bar > 5 Number of primary bloodstream infections (PBIs) per 1,000
central line days:
Numerator: Number of PBIs
Denominator: Total number of days a central line is in place
for all patients having central lines
X-bar and S chart n-bar > 10 Mean (average) time to initial antibiotic administration:
(Sum of each patient’s number of minutes between time of
physician’s order to initial antibiotic administration time)
divided by (Total number of patients receiving initial antibiotic
dose)
XmR chart Single measurement Number of referral calls per month:
A patient’s daily temperature graph
Numerator = top part of a fraction; denominator = bottom part of a fraction; mean = sum of all the observations divided by the number
of observations.
Sidebar 3-2.
A Case Example
A director of housekeeping wanted to know how her staff was performing. Knowing that time is easily measured and wanting
information about the length of time it takes to properly turn over a room between patients, the director began collecting
data about how long it should take and how long is too long.
After discussing issues of time with her staff, the director set expectations and established control charts showing upper and
lower limits. She asked and obtained staff buy-in and agreement with the upper and lower limits, which are data-driven, not
user-set. Workers were tracked for four weeks, and their daily average turnover times were charted. This exercise provided
information that replaced subjective impressions. The director was able to target outliers and focus on roadblocks to a quick
turnover. From this investigation, she was able to focus on improvement. Because the director quantified goals for her staff
that were objective and understandable and involved staff members in measure development, staff members felt empowered
to improve their performance. Staff members were clear about expectations and were not defensive about being held account-
able for improving performance.5
Multiple levels of comparisons are possible and even desirable. Comparison methodology can use the raw data from an organi-
Organizational performance can be compared to that of like zation’s observed measurements, or it can incorporate risk-adjust-
organizations (those of the same size that offer the same or simi- ed data for fairer comparisons. Not all measures require risk
lar services), to organizations in the same market share location adjustment, but typically health outcome measures should be
(competitors), to organizations in the same state or region, or to adjusted for each patient’s severity of illness. To level the playing
organizations on a national or even an international level. Many field in comparing outcomes, certain patient factors associated
health care organizations are very interested in how their com- with poor outcomes should be adjusted because these factors may
petitors are performing because in today’s world, customers can be beyond the control of the health care provider.
go virtually anywhere for health care—even the Internet.
45
Figure 3-8. Types of Control Charts

This chart shows the four most common types of control charts: the p-chart (for proportion measure), the u-chart (for ratio measure),
the X-bar and S chart (for continuous measures), and the XmR chart (for small-sample continuous data).
p-chart
Proportion
u-chart
Ratio
X-bar and S chart

Mean
Standard Deviation
XmR chart
(such as individual
being measured
Unit of what is
body temp.)
46
These factors can include, but are not limited to, the following: Percentiles
• Patient’s age A percentile is a numbering statistic indicating the relative
• Sex position of an individual score per 100 scores, arranged in the
• Comorbidities order of magnitude. Percentiles offer another approach to
• Death quantify the spread of data in a distribution. Here, the summa-
• Complications ry rates for proportion or ratio measures from many entities
• Length of stay are considered examples of continuous data that can be plot-
• Cost7 ted as a histogram. The median data point is equal to the 50th
percentile (which divides the data set in half ).
To risk adjust data, organizations can use the following methods:
• Stratify the measure into one or two major risk factor cate- Sometimes the spread of a distribution is characterized by
gories, such as age, sex, or individuals with diabetes and specifying the 10th and 90th percentiles to remove extreme
individuals without. data points on either side of a distribution. For the high school
• Use statistical models, such as logistical regression tech- class rank example, imagine that 530 students are divided into
niques, involving multiple patient risk factors. 100 blocks with approximately 5 students in each percentile.
The top five ranked students would be in the first percentile.
Some measures do not require risk adjustment for fair compar- Meghan, who ranked 30th, would be in the 6th percentile
isons. For example, a continuous measure that looks at the block of the distribution for her graduating class—meaning
time from emergency department arrival to thrombolysis that 6% of the students have a higher GPA than hers (because
administration for acute myocardial infarction (AMI) patients the GPAs are ranked from highest to lowest). Meghan’s per-
having elevated ST segments or left bundle branch blocks on centile rank is calculated as follows:
their initial electrocardiograms would generally not be risk
adjusted. This process measure looks at how effective organiza- Student’s rank × 100 30 × 100
tional policies and procedures are concerning the administra- = = 5.65, or 6th percentile
No. of students + 1 531
tion of thrombolytic medication to AMI patients who have it
prescribed. It does not matter whether the patient is 35 years Frequently used percentiles are quartiles (25th, 50th, and 75th
old or 65 years old, whether the patient is diabetic or a smok- percentiles). It is often helpful to look at the 25th to 75th quar-
er, or whether the patient has other extenuating circumstances. tiles (the middle half of an ordered data set, called the “interquar-
tile range”) to assess the overall spread of a distribution.
Methods of Comparison Analysis
Comparison data are commonly reported as summary data in Z-Scores for Outlier Performance
tables as rows and columns of numbers. To make data compar- A Z-score tells how many standard deviations from the mean
isons easier to understand, the following presentation methods a particular measurement value or score is. Because a Z-score
are often used: is based on the mean and standard deviation calculations, it
• Rank order can be a more precise evaluation tool for detecting outlier per-
• Percentiles formance than medians and percentiles.
• Z-scores for outlier performance
• T-statistics The Z-score is based on a standardized normal distribution
• Confidence intervals where a unique measurement value, such as a health care organi-
• Various graphical displays (see pages 38–44) zation’s observed rate of performance, is transformed into a value
that portrays where that organization’s level of performance lies
Rank Order in relation to the overall group mean. If the organization’s Z-
When objects are ranked, they are placed in the order of their score equals zero, its level of performance is the same as the
relative position, which is commonly from the highest to low- group’s level of performance. An organization which has a Z-
est score or from the best to the worst performance. For exam- score that lies in either tail (the area approximately beyond –3 or
ple, a high school’s graduating class members are typically +3 standard deviations) is considered a statistical outlier because
ranked from the student who has the highest grade point aver- the organization appears to be performing far better or far worse
age (GPA) to the student who has the lowest. than others (depending on the direction of improvement for the
47
measure). The following formula shows how to calculate a Z- represents the state mean, which was calculated as the average
score for continuous measurement data (n > 10): of the performance scores of all hospitals within that state; the
state’s rate was 0.72. To find performance that is statistically
Observed – Expected × √ n
Z= significant on this report, one needs to look for any hospital’s
Standard deviation
confidence interval (that is, horizontal bar) that does not
T-statistics include either the national or state rates (that is, does not cross
The t-statistic is a standardized score that is used in place of the one or both vertical bars). From this report, then, Hospital A
Z-score for measures that have a small number of observations appears to be demonstrating performance excellence, because
or sample size (n). Whereas a Z-score is a single distribution, its range of performance is beyond that of both the national
there are many t distributions—in fact, there are different dis- and state comparison rates. Hospital B seems to be demon-
tributions for different sample sizes. strating substandard performance; its range of performance is
inferior to both comparison rates. Finally, Hospital C’s per-
Confidence Intervals formance is similar to that of both the national and state aver-
A confidence interval establishes a range and specifies the ages; its range of performance coincides with these rates.8
probability that the range encompasses the true population
mean. For instance, a 99% confidence interval (approximate- Verifying Significance
ly) is calculated by taking the sample mean plus or minus 3 After an organization collects and analyzes data, it should
standard errors of the mean. The standard error of the mean is determine whether the results are statistically significant—that
obtained by dividing the standard deviation by the square root is, whether the data are in fact meaningfully related and the
of n (number of observations). patterns observed are not simply the result of chance.
Researchers measure the significance level of what they are test-
Whereas the standard deviation is used to describe the variabil- ing. For example, a significance of p < 0.05 means that there is
ity of the measurement values, the standard error is used to a 1 in 20 chance that the relationship between the variables
draw inferences about the population from the sample that has being tested is a result of chance. The level of significance pro-
been taken. A larger sample of observations will have a small- vides information about how likely it is that the person per-
er standard error than those with a smaller sample from the forming the data analysis has made an error or the wrong
same population. assumption. Generally, good research tolerates a 0.05 error
rate, or a 5% chance of having made an erroneous connection.3
With a larger number of observations, more statistical preci- To help calculate statistical significance, an organization can
sion or power is added to data analysis—allowing for more use computer software programs. Any statistics textbook can
meaningful interpretations and use of the data. Cautious inter- also help with this.
pretation of results derived from small populations is necessary
to prevent erroneous conclusions being drawn. Using standard Presenting Analyzed Data
deviation and standard error, which are based on sample size We have all heard the real estate industry’s mantra, “location,
calculations, helps to curtail data analysis mistakes. location, location.” In statistical analysis, “presentation, presenta-
tion, presentation” is a key part of data interpretation and use.
Figure 3-9 on page 49 shows a sample report format that uses Data presentation involves deciding on the best display format so
a 95% confidence interval to present outcomes for hospitals in that the data tell their own story and are readily usable and inter-
one state. Patients were surveyed at discharge to rate each hos- pretable by those who make decisions and monitor performance
pital’s level of performance relative to coordination of care. progress. The data must be displayed in formats that are con-
The hospitals were listed in alphabetic order, with a black cir- ducive to further analysis to identify opportunities to improve
cle denoting a hospital’s score and a horizontal bar indicating care and to examine the results of improvement efforts.
its confidence interval around the observed rate of perfor-
mance. Two vertical bars provide comparison functionality. Sometimes raw data are partially processed and displayed in a
The first vertical bar represents the national average rate devel- tabular format in reports. Although these “table data” may be
oped from national comparative databases over a rolling two- of some use in understanding an organization’s performance,
year period; the national rate was 0.71. The second vertical bar those responsible for using and understanding the reports
48
Figure 3-9. Sample Report Format for Confidence Level Comparisons: Coordination of Care Comparisons
The horizontal lines show the patient satisfaction rates for individual hospitals. The vertical bars indicate the national and state com-
parison rates. A hospital’s performance is statistically significant if its horizontal line does not intersect one or both of the vertical lines.
often take those numbers and display them in graphical charts Each organization must decide which analysis and presenta-
to make the data more user friendly and easier to interpret. tion styles are best suited to its information and performance
Several approaches may be used to create such graphical charts, improvement needs.
including paper and pencil, common spreadsheet packages,
and statistical software packages. Looking Holistically at Performance Measures
To get a true picture of an organization’s performance, all of the
When presenting measurement data to others, it is very impor- organization’s performance measures should be included in one
tant to clearly explain what a measure is, how its population is report that presents measure data trended over time. This
defined, who is excluded, and whether the measure is risk approach allows the user to evaluate performance holistically. It
adjusted. Proper explanations are imperative for the data to be also does not give a false sense that because everything in one
used to make informed decisions. Data analysts need to be able area is fine, this is true for the entire system. An example to illus-
to manipulate and interpret performance data and present trate this approach would be the data used for three reports to
them in a way that engages and provides insight to others.9 the board required by Joint Commission standards for many
health care organizations. The first report comprises human
Various Graphical Displays resources data; the second looks at the environment of care; and
There are many ways to present comparison data graphically. the third addresses performance improvement. The environ-
Graphics—including bar charts, line charts, pie charts, tables, ment of care and human resources reports state that everything
and control charts—can add more interpretive value to data is fine in the organization. However, the performance improve-
than a tabular format. A combination of graphs and tables, ment report notes that there have been equipment-related
used together in a complementary fashion, lets the user have at employee injuries. In addition, there is no education program
hand all the available information for making evidenced-based scheduled to address the proper use of equipment. This example
decisions or for deciding whether further investigation is war- illustrates the need to evaluate all data at the same time to avoid
ranted. When presenting data for use and interpretation, users a false sense that all systems and processes are working well
should understand the following points: together and are integrated.
• Who collected the data
• How the data were collected Some of the most common ways to report performance measures
• When the data were collected together include using balanced scorecards and dashboards.
• Where the data were collected
• What these values represent
49
Balanced Scorecards For balanced scorecards to be effective, organization leadership

The balanced scorecard is a performance management tool that must be involved in their development, implementation, and
is used to provide an integrated perspective of an organization. continued use. In addition, employees should be aware of the
Developed by Kaplan and Norton, it is based on the concept scorecards and what they mean.12
that organizations must evaluate and manage their business
from the following perspectives: Dashboards
• Customer One of the important ways that computers have influenced the
• Internal business process analysis and reporting of data is through the creation of soft-
• Growth and learning ware that displays data in an easy-to-read, easy-to-understand
• Financial10 format. Modeled after an automobile dashboard or an airplane
cockpit, visual dashboards can display everything from a
Some metrics to include in a balanced scorecard include the patient’s vital signs to the functional health of multiple infor-
following: mation systems in a hospital to network security. Clinicians
• Average length of stay can use dashboards to examine patients’ conditions, senior
• Maintained bed occupancies executives can use dashboards to monitor progress toward
• Full-time employees per adjusted occupied bed business goals, and network administrators can use dashboards
• Case-mix index to monitor electronic wireless networks, continually checking
• Monthly surgical cases, both outpatient and inpatient for unauthorized access.13
• Inpatient and outpatient reviews
• Cost per adjusted patient day Dashboards are valued for their ability to synthesize informa-
• Percentage of revenue from charitable sources tion and to condense large amounts of performance data into
• Revenue and expense per physician a limited number of key indicators.14 Typically, dashboards are
• Margin per department tied to an organization’s balanced scorecard, which funnels all
• Admitting-process performance11 clinical, financial, and administrative information into one
place. Dashboards can also pull information from a variety of
A balanced scorecard is an organizational framework and tool for systems as well as data entered manually into the application.
describing, implementing, and managing strategy at all levels in
an organization. It translates organization strategy into terms that Organization can use dashboards to provide information to
can be easily understood, communicated, and acted upon by evaluate their current performance and also the impact of var-
people inside and outside the organization.12 Balanced scorecards ious initiatives on the measured indicators. For example, the
provide a framework for unifying an organization around com- impact of a new enterprisewide program to improve patient
mon goals and consistent definitions and measurements.11 safety can be evaluated by using clinical quality indicators,
patient satisfaction data, employee satisfaction data, and finan-
Balanced scorecards also allow organizations to standardize cial indicators—all of which are housed on a dashboard. The
certain performance indicators across all departments and most common indicators used in health care executive dash-
locations. This facilitates the equal comparison and analysis of boards include the following:
business processes that are common to different departments.11 • Clinical quality indicators
Scorecards also include benchmarks to which organizations • Customer service indicators
can compare themselves to verify performance and identify • Staff indicators
areas for improvement. • Finance and resource indicators
• Patient activity indicators
The advantage of a balanced scorecard is that it encourages • Unit-specific indicators14
cross-functional communication and produces a common, eas-
ily understood language with respect to organizational objec- Because of the immense amount of data housed in a dash-
tives and performance measures. It allows visualization of an board, most such systems are automated. Dashboard metrics
organization’s accomplishments and identifies areas of excel- are color coded—usually green, yellow, and red—and can alert
lence through best practices.12 managers to areas that need improvement or attention. With
access to a dashboard, an organization leader does not need to
50
Sidebar 3-3.
Report Cards
Report cards can be used to clearly illustrate to the public the performance of a health care organization. They contain infor-
mation on the quality of care, patient access to and satisfaction with care, and the financial performance of an organization.
They compare organizations against similar organizations and against a standard of evidence-based care, and they are open to
the public.
Administrators should take the data from these report cards very seriously and use this information to communicate with
the medical staff.5 Following are some possible actions to take in response to report cards:
• Review the indicators in the report.
• Rank the measures based on their priority to the institution.
• Understand the results, including the indicators’ exclusion/inclusion criteria and statistics.
• Evaluate the data for each relevant indicator.7
file a request for someone to pull together a special report. It Individual Training Opportunities
offers real-time, clear information about potential problems. Individuals who analyze health care data in an organization
setting may obtain training from various sources. Some learn
It is important when designing a dashboard to have input from on the job from experienced colleagues or through trial and
organization leadership. The dashboard should reflect the val- error; others may attend workshops or other educational sem-
ues and mission of the organization and the goals of organiza- inars. In addition, a number of journals and books related to
tion leadership. performance measurement contain information about per-
formance improvement measures and data analysis methods.
See Sidebar 3-3, above, for information on report cards. (Sidebar 3-4 on page 52 lists selected performance improve-
ment journals and newsletters.) Education and certification in
Training in Data Analysis quality management techniques are also available.
The Joint Commission’s accreditation manuals include stan-
dards that apply to training in data analysis and statistical The following are some examples of how to become more pro-
tools. “Leadership” (LD) standards state that an organization’s ficient in the analysis, interpretation, and use of data:
leaders need to set expectations, develop plans, and manage • Take an introductory course in statistics or statistical process con-
processes to measure, assess, and improve the quality of gover- trol. Most colleges offer basic statistics courses, and some even
nance, management, clinical, and support activities. Staff offer courses that focus on health care–related statistics. Many
training in performance improvement approaches and meth- colleges and junior colleges offer courses in statistical process
ods is also necessary. control theory, which may be found in the mathematics, busi-
ness, manufacturing, or engineering sections. The basic foun-
“Improving Organization Performance” (PI) standards require dation for using and interpreting control charts is identical,
organizations to use appropriate statistical techniques to ana- regardless of the field of application, and can be adapted to
lyze and display data. The intent of the latter requirement is health care settings with a little creativity and thought.
that understanding statistical techniques is helpful both in
assessing the nature of variation and in identifying a process • Attend educational seminars offered by national/local profes-
that needs to be improved. By understanding the type and sional affiliations. Many professional societies, not-for-profit
cause of variation through the use of statistical tools and meth- organizations, and for-profit organizations offer educational
ods, an organization can focus its attention and resources on seminars on performance measurement in health care, cov-
specific areas that show room for improvement. ering the use of performance improvement tools, including
control charts and comparative data analysis.
51
Sidebar 3-4.
Quality Improvement Journals and Newsletters
American Journal of Medical Quality Journal of Clinical Outcomes Management

http://ajm.sagepub.com http://www.turner-white.com/jc/jc01.php
The Joint Commission Benchmark® Quality Digest

http://www.jcrinc.com/26813/newsletters/30/ http://www.qualitydigest.com
The Joint Commission Journal on Quality and Patient Safety Quality & Safety in Health Care
http://www.jcrinc.com/26813/newsletters/32/ http://qshc.bmj.com
International Journal for Quality Health Care Quality Management in Health Care
http://intqhc.oxfordjournals.org http://www.qmhcjournal.com
Journal for Healthcare Quality

http://www.nahq.org/journal/
• Obtain certification. The title Certified Professional in operation of quality control systems, application and analy-
Healthcare Quality (CPHQ) is offered by the National sis of testing and inspection procedures, the ability to use
Association for Healthcare Quality (NAHQ) to individuals metrology and statistical methods to diagnose and correct
who pass a written test on health care quality and improve- improper quality control practices, an understanding of
ment issues. The CPHQ examination is available on an ongo- human factors and motivation, facility with quality cost
ing basis at more than 100 computer testing centers located concepts and techniques, and the knowledge and ability to
throughout the United States. Training materials and guide- develop and administer management information systems
books for the exam are available from http://www.cphq.org. and to audit quality systems for deficiency identification
and correction.
In addition the NAHQ, the American Society for Quality 3. Quality Auditor. A Certified Quality Auditor is a profes-
(ASQ) offers information related to statistical process control sional who understands the standards and principles of
for skilled quality professionals as well as for beginners. auditing and the auditing techniques of examining, ques-
Traditionally, ASQ’s focus has been on the manufacturing tioning, evaluating, and reporting to determine a quality
industry, but that focus has expanded to include the service system’s adequacy and deficiencies.
industry, including health care. ASQ offers 14 certifications. 4. Six Sigma Black Belt. A Certified Six Sigma Black Belt is a
Although each has advantages, 6 have strong applications in professional who can explain Six Sigma philosophies and
health care, as follows15: principles, including supporting systems and tools. (For
1. Manager of Quality/Organizational Excellence. A Certified more information on Six Sigma, see Chapter 4.)
Manager of Quality/Organizational Excellence is a profes- 5. Six Sigma Green Belt. A Six Sigma Green Belt operates in
sional who leads and champions process improvement initia- support of or under the supervision of a Six Sigma Black
tives. He or she facilitates and leads team efforts to establish Belt, analyzes and solves quality problems, and is involved
and monitor customer/supplier relations, supports strategic in quality improvement projects.
planning and deployment initiatives, and helps develop mea- 6. Quality Improvement Associate. A Certified Quality
surement systems to determine organizational improvement. Improvement Associate has a basic knowledge of quality
2. Quality Engineer. A Certified Quality Engineer understands tools and their uses and is involved in quality improvement
the principles of product and service quality evaluation and projects, but doesn’t necessarily come from a traditional
control, including, but not limited to, development and quality area.
52
Summary 5. Dlugacz Y.: Measuring Health Care: Using Quality Data for
Analysis of performance measurement data is a necessary part of Operational, Financial, and Clinical Improvement. San Francisco:
Jossey-Bass, 2006.
the improvement process, and some familiarity with statistics is
6. Benneyan J.C., et al.: Statistical process control as a tool for
necessary to perform that analysis effectively. The statistical tech-
research and healthcare improvement. Qual Saf Health Care
niques used do not need to be complex to be effective, nor is great 12:458–464, 2003.
variety needed. More important is knowing how to choose the 7. Dlugacz Y., et al.: QM Environment: The Meaning of Measures:
best tool to display data accurately and how to interpret data to Using Quality Management Indicators to Improve Patient Care. Lake
make informed decisions about improving processes. Success, New York: North Shore–Long Island Jewish Health
System, Center for Learning and Innovation, 2007.
A health care organization with leaders and staff members trained 8. Joint Commission on Accreditation of Healthcare Organizations:
Managing Performance Measurement Data in Health Care. Oakbrook
in proper performance measurement management techniques
Terrace, IL: Joint Commission Resources, 2001.
will be able to make data-driven decisions to improve perfor-
9. Spath P.: Move for measurement to data intelligence. Hosp Peer Rev
mance and provide better-quality care to patients. Organizations pp. 174–176, Dec. 2004.
can share their success stories and lessons learned through net- 10. Tarantino D.: Using the balanced scorecard as a performance
working, professional associations, or publications to help other management tool. Physician Exec pp. 69–72, 2003.
organizations to improve their systems as well. 11. Wyatt J.: Scorecards, dashboards, and KPIs keys to integrated
performance measurement. Healthc Financ Manage pp. 76–80,
References Feb. 2004.
12. Schwartz J.: The balanced scorecard versus total quality
1. Carey R.: Improving Healthcare with Control Charts. Milwaukee: management: Which is better for your organization? Mil Med
ASQ Quality Press, 2003. 170:855, Oct. 2005.
2. Hussain R.: Central Tendency. In Dlugacz Y.: The Meaning of 13. Briggs B.: Dashboards merge health data: Software vendors assert
Measures. Great Neck, NY: The Krasnoff Quality Management dashboard technology will improve monitoring of clinical and
Institute, 2007. administrative data. Health Data Manag 13(4):58, 2005.
3. Dlugacz Y.: The Quality Handbook for Health Care Organizations: A 14. Mazzella-Ebstien A., et al.: Web-based nurse executive dashboard. J
Manager’s Guide to Tools and Programs. San Francisco: Jossey-Bass, Nurs Care Qual 19(4):307–315, 2004.
2004. 15. American Society for Quality: Quality in Healthcare: Training and
4. Hussain R.: Standard Deviation. In Dlugacz Y.: The Meaning of Certification. http://www.asq.org/healthcare-use/training/
Measures. Great Neck, NY: The Krasnoff Quality Management overview.html (accessed Aug. 5, 2007).
Institute, 2007.
53
Using the Data Management Process to

Drive Performance Improvement
Chapter 4
A fter data are collected and analyzed, it is important to

respond to the data. Although this seems obvious,
many organizations collect and analyze data and do nothing
To begin, organization leaders should consider the processes
that are high risk, high volume, or problem prone. Some of
the other criteria commonly used by health care organiza-
more, missing opportunities for performance improvement tions to help prioritize projects include the following:
and quality enhancement. • Impact on the customer
• Need to improve
Prioritizing Areas for Improvement • Urgency of the improvement need
Responding to data is sometimes easier said than done. • Relationship to the organization’s strategic plan
Given the volume of data analyzed in an organization and • Frequency of occurrence
the resources available for performance improvement, most • Probability of success
organizations need to prioritize performance improvement • Financial impact
efforts. Leaders must systematically identify the critical areas • Leadership interest
where outstanding performance is required if the organiza- • Effect on patient outcomes
tion is to accomplish its mission. Otherwise, improvement • Physician satisfaction
efforts create outcomes that don’t contribute much to the • Regulatory requirements
goals and mission of the organization. A successful prioritiza-
tion process is the foundation for a successful performance It is worth noting that the probability of success for a per-
improvement effort. Leaders are well advised to commit the formance improvement endeavor depends heavily on leader-
time necessary to thoughtfully and carefully consider their ship’s support and skill in managing the performance
options. improvement process. Thus, it makes sense for the evaluation
criteria to include leadership commitment to the project, the
57
availability of a competent team leader and facilitator, the abil- • Only one person should be talking at a time. Do not interrupt
ity to engage the medical staff (if appropriate), and the release someone who is offering an idea.
of employees to work on the project. These, more than any
other factors, may influence the number of projects that can be Effective brainstorming can be accomplished in the following
undertaken at a given time. five basic steps:
1. Define the subject of the brainstorming session. This ensures
One way to prioritize improvement efforts is to begin with a that the session will have direction and that ideas will not
list of all issues, problems, concerns, or potential improvement be too diffuse to be useful. Make sure everyone understands
projects, as generated by the data. Looking at supporting data the focus before ideas start flowing.
when prioritizing improvement projects restricts the number 2. Think briefly about the issue. Allow enough time for team
of issues to be considered to those truly worthy of considera- members to gather their thoughts but not enough time for
tion and avoids wasted discussion time. detailed analysis. Do not give group members time to sec-
ond-guess their ideas. Be aware that self-censorship will sti-
Tools for Prioritization fle creative thought.
Organizations can use several tools to help prioritize issues. 3. Set a time limit. Allow enough time for every member to
Following is a discussion of some of the most common tools. make a contribution, but keep it short to prevent prema-
ture analysis of ideas.
Brainstorming 4. Generate ideas. Leaders can use a structured format or an
Brainstorming is a structured but creative process that a group of unstructured format. In a structured format, group mem-
people use to generate as many ideas as possible in a minimum bers might express ideas by taking turns in a predetermined
amount of time. This process stimulates creativity and encour- order, with the process continuing in rotation until either
ages many perspectives on an issue. If properly conducted, brain- time runs out or ideas are exhausted. In an unstructured
storming provides a safe environment for people to express ideas. format, group members might voice ideas as they come to
mind.
If not monitored, a brainstorming session can quickly move to 5. Clarify ideas. The goal is to make sure that all ideas are
a blamestorming session, where participants identify issues and understood by the group, not judged or analyzed.
seek out people or things to blame for those issues. In some
health care organizations, people spend more time justifying Brainstorming is not an end in itself. The ideas generated
why performance is the way it is than looking at ways to through the process have to be assessed and refined. However,
improve that performance. Organizations that reward brainstorming is a start—a way of moving toward an identified
improvement, not merely measurement of data, eliminate goal. A repeat brainstorming session can be held after a couple
excuses and shift focus to how the organization will achieve days, in case people think of other ideas after the original meet-
better performance.1 The focus of brainstorming sessions ing. When the idea list is finalized, the group can review the
should be to identify potential priorities and solutions, not to list together and combine ideas, or they can strike out ideas
complain about problems and search for people to blame. that do not seem feasible. The remaining ideas can then be
ranked from most to least feasible for implementation.
When brainstorming, participants should keep in mind the
following ground rules: Pareto Charts
• There is no such thing as a bad idea. All ideas that address the Pareto chart analysis is the process of ranking opportunities to
subject at hand are valuable. The individual moderating a determine which should be pursued first. Such analysis is per-
brainstorming session should express this as part of his or formed by dividing the data into subsets to help locate con-
her introductory comments or briefings. tributing factors. Several layers of drilling down are often
• Quantity, not quality, is the goal. More ideas are better. Build required to help with prioritization. Some refer to this process
on, or “piggyback,” each other’s ideas. as “slicing and dicing” the data in different ways to get differ-
• Never criticize other people’s ideas. It is crucial that neither the ent pictures and possibly narrow down the location of the
leader nor the other team members comment on or react to problem area. Rational subsets are the logical groupings used
an idea. Remember that the best ideas are sometimes the to divide the data for further study and investigation of causal
most unusual ideas. factors.
58
Chapter 4: Using the Data Management Process to Drive Performance Improvement
Examples of rational subsets include the following: can immediately be used to guide action planning, program
• For acute care—different nursing units, day and night shifts, developing, or evaluation and measurement. The concept
weekends/holidays and regular work week, patient popula- mapping process involves the following steps:
tions by age, physician/surgeon groupings, and different 1. Define the focus of the mapping project.
patient classifications (such as cardiovascular, renal, or neu- 2. Select participants for the process and determine logistics
rological) and schedule.
• For behavioral health care—drug/alcohol abuse or various 3. Brainstorm ideas about the process.
mental illness classifications 4. Synthesize and sort the ideas.
• For ambulatory care—general anesthesia or intravenous 5. Rate the ideas for one or more variables of interest, such as
sedation relative importance or feasibility.
• For home care—home medical equipment, intravenous infu- 6. Conduct statistical analysis to generate maps.
sion therapy, home health care, personal care, or hospice 7. Use the maps and associated results to address the purpose
• For long term care—Medicare, Medicaid, or private payers of the project.4
To further analyze data resulting from drill-down analysis, the Causal Loop Diagrams
data can be displayed in run charts, control charts, line graphs, A causal loop diagram is a modeling tool that links several vari-
bar graphs, pie charts, and so on. ables with their causal relationships. This tool can be used to
document internal and external driving forces that affect spe-
Cause-and-Effect Diagrams cific variables. It can help identify gaps in learning and the
A cause-and-effect diagram (see Figure 4-1 on page 60), also called need for additional data. After processes have been implement-
a “fishbone diagram” or an “Ishikawa diagram”—named after its ed, causal loop diagrams can help verify that new processes will
inventor, Kaoru Ishikawa—is a tool that can illustrate various not adversely affect existing ones.
factors that have an impact on a particular outcome.2 It can help
identify the root cause of a problem, especially a complex prob- Affinity Diagrams
lem. Oftentimes, organizations use fishbone diagrams in root An affinity diagram is a structured tool that a team can use to
cause analysis (RCA) and failure mode and effects analysis develop a short list of items quickly. It combines ideas into sim-
(FMEA) exercises. (For more information on RCA and FMEA, ilar groups, resulting in a manageable number of topics. One
see pages 67 and 66, respectively.) This type of diagram is often approach to creating an affinity diagram begins with brain-
used to display the main process inputs (“large bones”) for a storming all the potential criteria that can be used to evaluate
selected performance measure to determine the likely causes of performance issues. Each participant does this individually, in
poor or unstable performance. Brainstorming sessions involving silence, writing each idea on a sticky note. After all participants
staff members who are actually involved in the day-to-day per- have recorded their ideas, each is asked to put the notes on a flip
formance of the measure help to identify other minor, support- chart and explain the meaning of each idea. After all ideas are
ing process inputs (“small bones”). posted, the group is asked to silently group the notes into logi-
cal categories. The categories are named and subsequently
Concept Mapping become the criteria for evaluation of issues.
The concept mapping methodology can be used to produce a
picture, or map, of the ideas or concepts of an individual or Mock Tracers
group. It is a schematic representation constructed from con- To trace the performance of care and identify and prioritize
cepts and/or linking words and arranged in a vertical hierarchi- potential areas for improvement, organizations can conduct
cal order, proceeding from the most abstract expression of a mock tracers. To do this, the organization selects a specific
concept to its multiple domains and relationships down to its patient and then “traces” the care provided to that patient from
empirical indicators.3 Designed to integrate input from multi- admission to discharge, carefully examining every step along
ple sources with different content expertise or interest, concept the care continuum and the relationship of systems for provid-
mapping combines group discussion activities, such as brain- ing care. The tracer methodology is a cornerstone of The Joint
storming, sorting, and so forth, with statistical analysis. The Commission’s accreditation process. (For more information on
resulting concept map visually depicts the thinking of an indi- the tracer methodology and conducting mock tracers, see
vidual or a group and constitutes a framework or structure that Chapter 5.)
59
Figure 4-1. A Cause-and-Effect Diagram
A long term care organization that has a problem with high rates of weight loss among its residents might generate a cause-and-
effect diagram that looks something like this for the causes found. Each bone in the diagram can contribute to the cause(s) linked
to the weight loss problem.
Multivoting rating when only one or two participants find it important. A

One helpful tool for managing the priority-setting process is critical last step in multivoting is to ask the group, “Does this
multivoting, which is used to systematically achieve consensus list make sense for us?” The group must achieve consensus on
on a condensed list of important items. Multivoting, or multi- the results before the list is finalized.
ple voting, is a group decision-making technique that is
designed to reduce a long list (suggested actions, improvement Prioritization Matrixes
priorities, and so forth) to a shorter list. It can be used after A prioritization matrix helps an organization define its clinical
brainstorming sessions to narrow down the list of original and operational goals and establish its priorities.5 It is a form of
ideas. convergent thinking that is used to reduce the number of con-
sidered opportunities to a realistic, manageable size. Its pur-
Within the multivoting process, each participant is allocated a pose is to provide a structured, data-driven approach to deter-
fixed number of votes that can be applied to the list of issues. mining what improvement opportunities are most likely to be
The votes may be allocated in any way the individuals wish— successful and to affect the overall organization in a meaning-
for example, an individual may allocate five votes to one issue ful way. The matrix has three general components: (1) the
he or she feels is extremely important, or one vote to each of issues or topics being considered, (2) the criteria used to prior-
five issues he or she feels are equally important. The votes are itize the issues, and (3) the rating scores for each issue.
tallied for each issue, and the top issues are retained, according
to a predetermined limit. The process may need to be repeat- Creating and completing a prioritization matrix does not need
ed to achieve a manageable list. to be a quiet process. In fact, the strength of the prioritization
matrix is not as a paper tool but as a focus for leadership dis-
A note of caution is warranted about the use of multivoting. cussion. After issues are considered and rated, rating scores
Because of the distribution of votes, an issue may receive a high must be tallied. This may be done in a variety of ways. The
60
individual scores of each participant may simply be added, or Leaders can use data to determine the appropriate team com-
the scores may be averaged. The scores are then added across position. Qualitative data about potential team member skills
all criteria, and a single final score is assigned to each issue. The and abilities can be collected from performance evaluations,
top issues are considered against the final criteria: Does this list training records, and human resources files. It is useful to
make sense for us? develop a description of the desired characteristics of the over-
all team. This list is then used to ensure that all requirements
Weighted Prioritization Matrixes are represented by the final team. Some of the characteristics
Often, not all evaluation criteria are of equal weight. For exam- that are typically required include the following:
ple, alliance with the organization’s strategic plan may be more
• Content knowledge. Technical knowledge and expertise are
important than the frequency of occurrence. In such a case,
necessary to describe the process as it is and as it is desired.
criteria may be assigned relative weights. The weights are then
Those who interact, perform, or experience the results of a
used as multipliers for the scores on each criterion, and these
process are in the best position to provide information about
weighted scores are used for the final tally. Examples of a
how it currently functions and often have substantial sugges-
simple prioritization matrix and a weighted prioritization matrix
tions for improvements. Each aspect of process performance
can be found in Figures 4-2 and 4-3 on pages 62 and 63,
should be represented. For example, if the team is to reduce
respectively.
the time needed to obtain an electrocardiogram in the emer-
gency department, representatives from cardiology, the
Preparing for Improvement emergency department, and the order-entry system should
Before getting too far along in the performance improvement be included. Or, if the team is to reduce the amount of time
process, organization leaders need to prepare for the process, required to perform intake in an adolescent behavioral
including creating performance improvement teams and allo- health care setting, representatives from admissions, social
cating necessary resources. work, nursing, and counseling should be included, as should
individuals who are knowledgeable about payer require-
Creating a Performance Improvement Team
ments. Sometimes it is helpful to use a flowchart of the
The most successful performance improvement efforts involve process to ensure that all aspects of the process are represent-
a team approach, which includes collaboration and consensus ed in the team’s composition.
building. Teams can cross disciplines, departments, and func-
tions. Teams are critical when prioritizing areas for improve- • Skill in performance improvement. Analyzing data, using per-
ment, as they ensure that expertise and objectivity come from formance improvement tools, contributing effectively to
many sources. In addition, teams foster a sense of commitment group discussion, and communicating clearly are all key
from participants and ensure that members take ownership of skills for performance improvement team members. In addi-
a project.2 See Sidebar 4-1 on page 64 for common character- tion, a team leader and facilitator must be selected. Whereas
istics of successful teams. Sidebar 4-2, also on page 64, pro- the content knowledge of the facilitator will be focused on
vides information on team charter statements. team process, the team leader must possess skill in team
management and expertise in the content area.
The exact composition of a performance improvement team is • Commitment to the process and a passion for improvement.
based on the knowledge, skill level, and commitment required Team members must be committed to the performance
for the effort to be successful. An improvement team should improvement process and possess a passion for improving
not be selected for political or personal reasons, but rather for the process under study. The team leader must be personal-
the unique contribution each member can make to the overall ly committed to creating improvements through teamwork.
results.
• Access to resources. At least one team member must have the
The most carefully chosen team can fail if consideration is not power and access necessary to gain resources for the team.
given to the basic elements that make a team successful. Leaders should specifically identify a team member who will
Leaders are wise to set up a team to succeed by creating an have access to administrative input when needed and who can
environment for the team where the focus is on organization- request the resources that the team needs to achieve its goals.
al improvement, not on solving problems within the team. • Ability to empower change. Team members should be selected
for their skill and expertise, but equally important is each
61
Figure 4-2. A Simple Prioritization Matrix
This type of diagram can be used to rate each issue or project based on an organization’s evaluation criteria. The final issues to be
considered are listed on the left side of the diagram, and the criteria for evaluation are listed across the top.
member’s sphere of influence. Influence is a necessary com- is true that interdisciplinary teams are necessary to solve
ponent of engendering collaborative, interdisciplinary process problems that cross several departmental lines. But
change, where no one individual has the authority to force that is not to say that work groups of the same discipline—
others to comply with recommendations. Influence is differ- sometimes called “natural work groups”—cannot also manage
ent from power; influence implies voluntary change on the performance improvement. Indeed, natural work groups can
part of others. Gaining a wide sphere of influence is neces- and should apply performance improvement strategies to their
sary for widespread change to be achieved. For this reason, it own work as part of their daily activities.
is often desirable to be sure that the team has a balance of
managers and frontline-level employees as members. Bringing together an interdisciplinary team not only ensures
that various disciplines involved in a process have some input
• Diversity of membership. The strength of teamwork lies in the
into its revision but also increases accountability because every-
ability to gain multiple perspectives on an issue or a prob-
one on the team acknowledges what is expected.2
lem. This strength is lost if the group is homogeneous. A
diverse team represents different backgrounds and encour-
The strength of a team is not automatically achieved when a
ages a focus on the problem from various perspectives. Each
group of individuals get together. Leaders provide the environ-
team member should supply some unique perspective or
ment and guarantee the characteristics that will encourage team
knowledge to the team as justification for his or her mem-
success, and they provide the resources to achieve team goals.
bership. Diversity in team membership also increases the
number of potential solutions considered, which in turn
Clearly Identifying Team Goals
increases the chances that the best solution will be chosen.
Leaders cannot manage performance improvement without
clearly identifying and articulating the expected results.
For this last reason, leaders may have a tendency to favor inter-
Quantitative and qualitative data should be used to describe
disciplinary teams over work groups of the same discipline. It
62
Figure 4-3. Weighted Prioritization Matrix
Using the same issues and criteria as in Figure 4-2, this matrix assigns relative weights to all criteria, reflecting their importance
to the organization. This makes it easier to identify the highest priorities to be addressed.
the outcomes expected from those charged with creating the performance teams are empowered with an implicit expecta-
improvement. In addition, leaders need to define all terms so tion that they will save money and resources. In the long run,
that everyone involved in the improvement effort approaches this may be true. But in the short term, performance improve-
the issue with a common understanding. ment requires the allocation of resources. As with any other
organizational investment, an up-front investment is required
When establishing expectations for the team, organization to engender long-term, stable returns.
leaders should consider setting specific dates for progress
reports. Progress reports should focus on gains made in the Many organizational resources should be allocated to achieve
specific performance improvement project. Data should be performance improvement, including financial resources,
submitted to leadership along with representations of any tools information, staff time, and facilities and equipment.
used to analyze performance. In addition, progress reports may
describe progress made in team development and the process. Allocating Financial Resources to Support Team Decisions
Team decisions often involve the investment of financial
Committing Resources to Improvement Teams resources to successfully implement an intervention. Data are
With a little effort and a planned strategy, an organization can used to estimate the cost of the intervention, the expected value
achieve significant performance improvement, but not with- of the intervention, and the expected return on investment.
out the consumption of organizational resources. Many times, Although it is undesirable to give a team a charge to improve a
63
Sidebar 4-1.
Common Characteristics of Successful Teams
Successful teams have certain elements in common, and leadership can measure and monitor these elements to encourage
success. The characteristics common to successful teams include the following:
• Clear goals and objectives
• Clarity of each member’s role and expertise
• A standard process for team meetings and the team’s work
• Trained and oriented team members
• External support and recognition
• Effective leadership and facilitation
• Collaborative problem solving and decision making
• Presence of leaders who have the resources to implement proposed solutions
• Time for team meetings and assigned team work
Sidebar 4-2.
Team Charter Statement
The expected results of a performance improvement effort are often articulated as a team charter, or “charge statement.” The
charter statement is a written description of expected outcomes and typically includes the following elements:
• A statement that describes the issue or problem in its current state. This narrative includes a brief overview of the data that led
to the conclusion that an issue exists.
• An outline of the rationale for the quality effort. This section summarizes the reasons leaders agreed that the issue is a priority.
• The expected improvement. This section includes a detailed description of the expected outcome, including any quantitative
and qualitative measures that will demonstrate that the outcome has been achieved. Key terms and words are explicitly
defined. Leaders can express the expected results in terms of a desired rate or a range of rates. For example, the work of the
team could be to improve the process itself, eliminate rework, and reduce variability in practice.
• Scope of the team’s work. Leaders define how much work is expected from the team. It is just as important to identify what is
not expected from the group. Giving a team an unmanageably large project may lead to negative outcomes; focusing the
group’s work appropriately is key to success.
• Time frame. Results should be expected within a specified time frame; otherwise, the team may not progress.
• Resources. The resources available to the team and the process for requesting them are clearly identified.
• Reporting and communication. Expectations for reporting on progress and communicating results are clarified.
Leaders should reflect on the finished charter statement to be sure it describes realistic expectations and time frames. Many
times, leaders become frustrated by the slow progress of a team that doesn’t meet expectations, when in reality the expecta-
tions themselves are unrealistic, poorly communicated, or misunderstood.6 One way to ensure that expectations are clear is to
rely on data to describe the problem and expected outcomes.
64
process and no financial resources with which to accomplish it, it Providing Access to Information
is certainly acceptable to expect the team to justify an investment Teams need access to information about the process under study,
of financial resources. This justification ideally links a financial relative to both its current performance and the desired perfor-
return with a financial investment, but it may take other forms. mance. This information may include qualitative and quantita-
The following three types of financial justification data are useful tive data about process performance, patient/staff satisfaction
in evaluating and allocating financial resources: with the process, steps in the process, or outcomes of the process.
1. Cost–benefit analysis. Cost–benefit analysis involves the calcu-
lation of costs and returns expected for the planned interven- Frequently, teams find it difficult to access the data and infor-
tion. Both are expressed in monetary terms. Cost–benefit mation they need. Obstacles to gaining access to information
data are often expressed as a ratio, with benefits in the numer- may include fears about confidentiality, concern that “bad”
ator and costs in the denominator. A figure of greater than 1.0 performance will come under scrutiny, or resistance to the
is desirable. Examples might be costs of a patient education time required for collecting and interpreting data. Team mem-
program calculated against lost revenue from unplanned re- bers may lack statistical analysis skills or the ability to present
admissions or costs associated with hiring an enterostomal data in an understandable way.
therapist consultant in long term care against the costs of
healing a pressure ulcer. The performance improvement process is heavily based on data.
2. Cost-effectiveness analysis. Cost-effectiveness analysis, like As such, it requires a great deal of information for analysis of the
cost–benefit analysis, involves estimating financial costs current process and for monitoring the effect of any interventions
associated with the intervention. However, effectiveness applied. An organization must ensure that a team has the
may be measured in terms other than money. Effectiveness resources and cooperation it needs to gain accurate, complete
may include financial return but may also describe qualita- information if informed decisions are to be made.
tive returns, such as improved patient satisfaction or sup-
port of the organization’s mission. Effectiveness includes an Allowing Time to Conduct Meetings
assessment of whether goals were achieved and is generally A team must have adequate time to apply the performance
defined as answering the question, “Did the intervention improvement process, evaluate data, determine interventions,
work in a more effective way such that the investment was and oversee improvements. This may be the most difficult
justified?” Evaluation of cost-effectiveness data involves resource to apply in today’s hectic health care environment.
judgment and intuitive conclusions about the value of the Team membership is selected to minimize redundancy and
return on investment. maximize efficiency; it is imperative that team members are
3. Cost–utility analysis. Cost–utility analysis is the most com- consistently in attendance. Team members paid on an hourly
plex and difficult justification to measure. Costs are mea- basis must be paid for their participation and work time with
sured, as with the other two methods, in monetary terms. the team. All members must have plans to cover their work so
But utility is described as the contribution to society in they may attend meetings.
general, or as creating a desirable state. For example, utility
might be measured as a reduced number of institutional- In today’s geographically dispersed health care environment,
ized physically challenged children or a decrease in teen organizations may need to rely on virtual teams. For health care
pregnancy rates. Cost–utility is difficult to quantify, and networks in particular, it is often prohibitively expensive, in terms
many of the data used to justify the utility of a given inter- of both time and money, to have consistent physical attendance
vention may be qualitative, estimated, or projected. of all team members at centrally located meetings. Although they
are challenging, virtual teams can be successful. Virtual teams rely
The standards by which leaders judge the desirability of a given on technology, rather than travel, to get their jobs done, and they
financial investment should be applied consistently to per- must use whatever methods they can to keep in touch with each
formance improvement projects, as they would be to any other other and keep the momentum going. Videoconferencing, elec-
investment. Teams should know from the start what data and tronic real-time chat rooms, electronic bulletin boards, and
justification are needed to request financial investment, so they shared files may all be used to maximize communication while
can calculate and provide accurate information for leaders’ minimizing the number of physical meetings.
decision making.
65
Even virtual teams must have the time for an initial face-to-face tive technique that is used to prevent problems before they
start-up session. Virtual teams need training time for the tech- occur. It provides a look not only at what problems could
nology that will enable their work, as well as for the group occur but also at how severe the effects of the problems could
process. The sharing of information and solicitation of input for be. FMEA assumes that no matter how knowledgeable or care-
decisions becomes a top priority. Still, the human side of the ful people are, failures will occur in some situations and may
arrangement needs attention and periodic social time, and per- even be likely to occur. The focus is on what could allow the
sonal contact should be planned when resources are allocated. failure to occur, rather than on whom.
Committing Facilities and Equipment for Meetings and Ideally, FMEA can be used to help prevent failures from occur-
Communication ring. However, if a particular failure cannot be prevented,
With settings stretched to the physical limit, adequate, consis- FMEA then focuses on protections that prevent the failure
tent facilities for team meetings are often in short supply. A from reaching the patient, or, in the worst case, mitigate the
group needs to be able to consistently access a room that has effects of the failure if it reaches the patient.
support for the team’s work and tools, including a marker board
of some type, flip charts, writing surfaces, and accessibility. The Steps Involved in FMEA
The FMEA technique is based on studied engineering princi-
The team also needs the resources to communicate effectively. ples and approaches to designing systems and processes. It has
General clerical support for the team is ideal; this applied been successfully used in a number of industries, including the
resource allows the group to focus on solving problems rather airline, automotive, and aerospace industries. Varying by the
than on managing paper. Access to electronic forms of com- source consulted, FMEA can involve from as few as 4 to as
munication and streamlined communication approval process- many as 10 different steps. The approach described here has 8
es also foster successful transmission of monitoring data and key steps, as follows:
progress descriptions. 1. Select a high-risk process and assemble a team.
2. Describe the process.
Overall, resource allocation for teams should be planned dur- 3. Brainstorm potential failure modes and determine their
ing initiation of the team’s work and periodically reviewed for effects.
effectiveness. A variety of data elements should be submitted 4. Prioritize failure modes.
to leaders to ensure that improvement efforts are focused 5. Identify root causes of failure modes.
appropriately and progressing consistently. 6. Redesign the process.
7. Analyze and test the new process.
Defining the Problem 8. Implement and monitor the redesigned process.
After a team is formed, it should work collaboratively to identify
how performance will be improved and what improvement will When conducting an FMEA, a team should answer questions
look like. Many of the previously discussed tools—including such as the following:
brainstorming, Pareto analysis, and cause-and-effect diagrams— • What are the steps in the process? If this is an existing
can help with this process. A flowchart can also be helpful at this process, how does it currently occur and how should it
time. A flowchart uses symbols and words to show the steps, occur? If this is a new process, how should it occur?
sequence, and relationship of various operations involved in the • How are the steps interrelated? (For example, are they
performance of a function or process. By flowcharting a process, sequential or do they occur simultaneously?)
an organization can visualize all the areas of the process and deter- • How is the process related to other health care processes?
mine at which point improvements can be made. If the process • What tools should be used to diagram the process?
under study is complex in nature, with multiple subprocesses, it • What is the manner in which this process could fail? (When
may be beneficial for the team to break up the process and focus answering this question, team members should consider
on certain aspects to improvement performance. how people, materials, equipment, processes and proce-
dures, and the environment affect the process.)
Failure Mode and Effects Analysis • What are the potential effects of the identified failures?
A tool that can be helpful in studying a process is failure mode Effects of failures might be direct or indirect, long or short
and effects analysis (FMEA), a team-based, systematic, proac- term, or likely or unlikely to occur. The severity of effects
66
can vary considerably, from a minor annoyance to death or Redesigning the Process
permanent loss of function. In this part of the process, team When starting to redesign a process, a team should consider how
members should think through all the possible effects of a to ensure that the redesigned process addresses any issues or prob-
failure and list them for reference. lems effectively. Much time can be saved during the redesign
• What are the root causes of prioritized failure modes? What process by learning about what other organizations have experi-
would have to go wrong for a failure like this to happen? enced with the process in the past. An organization might want
What underlying weaknesses in the system might allow this to craft its performance improvement strategies based on existing
to happen? What safeguards (for example, double-checks) recommendations rather than “reinvent the wheel.”
are present in the process? Are any safeguards missing? If the
process already contains safeguards, why might they not Prior to commencing the redesign process, a team can turn to
work to prevent the failure every time? If this failure professional literature and conduct a search to gather process-
occurred, why would the problem not be identified before it specific information. Professional associations and organizations
affected a patient? focused on safety can provide a wealth of valuable information.
For example, the Joint Commission’s Sentinel Event Database
Root Cause Analysis offers several common root causes of patient safety issues and
Besides FMEA, another tool that can be helpful when further identifies improvement strategies that can help address these
identifying and defining a problem or process to study is root causes. Other sources for performance improvement strategies
cause analysis (RCA), which is a process for identifying the include the Institute for Healthcare Improvement, the Agency
basic or causal factors that underlie variations in performance. for Healthcare Research and Quality (AHRQ), and the Veterans
Variations in performance can (and often do) produce unex- Health Administration.
pected and undesired adverse outcomes, including the occur-
rence or risk of occurrence of a sentinel event. To redesign processes effectively, team members should focus
on systems rather than on people. Although there are several
Like FMEA, RCA focuses primarily on systems and processes, ways to redesign a process, the following are suggested strate-
not on the performance of a particular person. Through RCA, gies team members should consider:
a team works to understand a process or processes, the causes • Standardize. If the process is conducted consistently, whenever
or potential causes of variation, and process changes that make and wherever it is performed, there is less chance for error.
variation less likely to occur in the future. RCA is most com- • Simplify. Complex processes offer many points at which fail-
monly used reactively to probe the reason for a bad outcome or ures can occur. Many health care processes are unnecessarily
for failures that have already occurred. It can also be used to complex.
probe a near-miss event or as part of the FMEA process. • Optimize redundancy. If redundancy is built in, errors are less
likely to cause systemic failures that result in harm to
A thorough and credible RCA has several steps, which are sim- patients.
ilar to the steps in FMEA, as follows: • Automate. Technology can help avoid human error due to
1. Organize a team. stress, fatigue, and other human factors.
2. Define the problem. • Build in fail-safe mechanisms. If a failure does occur, a fail-
3. Study the problem. safe mechanism can stop the process before a sentinel event
4. Determine what happened. or near miss occurs.
5. Identify the contributing factors. • Document. Effective documentation and communication
6. Collect and assess data on proximate and underlying causes. can help prevent errors.
7. Design and implement interim changes. • Loosen coupling of process steps. It is important to allow more
8. Determine the root causes. time to detect failures or errors and more opportunity to
9. Explore and identify risk-reduction strategies. intervene to protect patients.
10.Evaluate proposed actions. • Tighten or loosen time constraints. It is important to optimize
11.Design, test, and implement improvements. the time between steps to minimize the chance of error or
12.Evaluate and communicate the results of improvements. failure.
67
• Increase detectability. A team should consider installing instruction—that tells how to solve a particular problem. An
alarms at critical points in the process. Even if a failure algorithm is specific, so there can be no doubt about what to
occurs, it can be detected and the associated error avoided. do next, and has a finite number of steps.
• Retrain or reinforce training. Staff members need to be aware
of issues that can lead to errors. Clinical Pathways
• Ensure team focus. Teams working together effectively com- Clinical pathways are guidelines that outline key interventions
municate and collaborate more efficiently, which can help and outcomes along a time line for a specific disease, such as for
spot and prevent errors. hip replacement or heart failure. A clinical pathway, or treatment
protocol, is based on a consensus of clinicians and includes only
When creating processes, team members should also consider those vital components or items proven to affect patient out-
the following: comes, either by the omission or commission of the treatment or
• Chance for success. How likely is the redesign element to suc- by the timing of the intervention. One type of clinical pathway
ceed in preventing a failure? is called a care map (see Figure 4-4 on pages 69–77). Care maps
• Staying power. How likely is the redesign element to effect a were initially designed to define patient flow and monitor length
long-term versus short-term solution? of stay. When a patient is initially diagnosed with a particular dis-
• Reliability. How reliable is the redesign element? Will it work ease or treatment need, he or she can be put on an appropriate
all the time? part of the time? care map, with the physician noting what should be accom-
• Risk. How likely is the redesign element to engender other plished in the patient’s daily plan of care. Care maps incorporate
adverse effects (“unintended consequences”) within the evidence-based guidelines as well as physician orders and clinical
process or another health care process? What would happen judgments. If a patient does not receive an expected intervention,
if this redesign element were not implemented? the reason for the omission is documented on the care map.
• Workability. How doable is the redesign element? Does the
organization have the resources to make the improvement A clinical guideline, such as a care map, incorporates evidence-
happen? based medicine with clinical, organizational, and financial
• Barriers to implementation. What barriers is this redesign ele- processes in the following ways:
ment likely to face during implementation? How receptive • Pinpoints gaps in the delivery of care through variations
will management, staff members, physicians, and others be from the standard
to the redesign element? What will be involved in eliminat- • Serves as a tool to promote coordinated care at the bedside
ing barriers to implementation? • Helps to direct the care toward evidence-based best practices
• Compatibility. Is the redesign element compatible with the • Enables analysis of organizational processes for improved
organization’s objectives and mission? efficiency
• Availability of resources. What will implementation of the • Provides a standard of care for varied patient populations with
redesign element cost, financially and otherwise? How much discipline-specific goals, focusing on patient and cost outcomes
staff time will be involved? Is the organization prepared to • Provides documentation that a standard of care was met
dedicate all the resources necessary to implement the project? • Allows for the prediction and planning of how to care for a
What type of training is involved in bringing staff up to specific patient population7
speed? Is the organization willing to invest in staff education? • Increases collaboration and efficiency by prospectively plan-
• Implementation time frame. How long will the redesign and ning for care
testing take? • Strengthens accountability by linking assessment and inter-
• Measurability and objectivity. How objective is the redesign ele- vention strategies with patient outcomes8
ment? How will improvement be measured and maintained?
Because care maps incorporate standardized guidelines as well as
As described in the following sections, organizations can incor- physicians’ orders and judgment, they can be tailored to meet the
porate a number of tools into a redesigned process. needs of an individual patient.5 For performance measurement,
many measures are created from points or steps in the clinical or
Algorithms critical pathways developed for specific patient populations. See
An algorithm is an ordered sequence of steps or instructions— Sidebar 4-3 on page 78 for information on dealing with staff
each of which depends on the outcome of the previous step or resistance to new initiatives, such as care maps.
68
Figure 4-4. A Care Map
This is an example of one organization’s care map.
(continued)
69
Figure 4-4. A Care Map, continued
(continued)
70
(continued)
71
(continued)
72
(continued)
73
(continued)
74
(continued)
75
(continued)
76
Source: The Krasnoff Quality Management Institute, a division of North Shore–Long Island Jewish Health System, Great Neck, NY. Used with permission.
77
Sidebar 4-3.
Dealing with Resistance
When introducing new initiatives, such as clinical pathways, critical practice guidelines, or care maps, organizations may
meet with some resistance. Some nurses may reject initiatives as meaningless paperwork, and some physicians may feel
that following a pathway, critical practice guideline, or map is tantamount to “cookie cutter” medicine. Leadership
should take the time to work with staff members to address their concerns and explain that evidence-based medicine
coupled with individual knowledge can yield the best possible care for patients. This may involve education sessions led
by individuals who champion the new process. In these sessions, specific data should be shared that show the success of
such guidelines or processes in other organizations and the direct and significant impact such evidence-based medicine
has had on clinical care and patient outcomes.
Clinical Practice Guidelines team plans specific interventions that will maximize driving
Clinical practice guidelines are promulgated by respected pro- forces and minimize restraining forces. Measurement of data
fessional societies, such as AHRQ, or are developed in-house representing forces in the analysis can be used to monitor the
to improve quality, utilization, and patient education.5 occurrence of obstacles to be considered when evaluating
Examples of clinical practice guidelines include guidelines for team progress.
pressure ulcer prevention and treatment and guidelines for car- • Pilot testing. Thanks to pilot testing, teams can implement
diac rehabilitation. Guidelines make evidence from aggregated the redesign on a small scale, monitor its function and
populations of patients available to physicians and standardize results, and refine the redesign, as necessary, without taking
practice to ensure quality care. They can improve treatment the risks associated with full-scale implementation. This
protocols and ensure the consistency of quality care across a helps ensure that the redesign is successful before significant
patient population. resources are committed to the redesigned process.
Testing and Implementing the Plan See the case study in Sidebar 4-4 on page 79 for an example
After a team develops a suggested initiative, team members that includes testing and implementing a plan.
should test the process to discover its workability. There are
several ways to test a new process, including the following: An important but often ignored aspect of process testing is to
• Paper testing. This involves conducting an FMEA or anoth- assess the unexpected effects of improvement efforts. In some
er similar activity on the new process on paper, including cases, modifying one part of a process to bring it under control
creating a revised flowchart. causes another part of the process to become more variable.
• Simulation. A paper, role-play, or computer-based simula- For example, efforts to reduce length of stay in a network’s
tion exercise can help a team anticipate the full range of acute care setting may result in increased numbers of home
decision-making and performance issues that might arise care visits or more intensive care needs for residents in long
with a new process design. term care settings. The testing process must be broad based
• Stress tests. It is helpful to play out how the new process will and must consider related processes as well as the process
function when several “worst-case” conditions or variables under study to make sure improvements do not result in unin-
converge at the same time. tended negative results elsewhere.
• Force-field analysis. Force-field analysis is a useful tool for
projecting the forces that will support implementation and Plan-Do-Study-Act (PDSA)
those that will hold back the improvement. It is a qualitative Although there are many effective ways to implement an ini-
analysis tool that helps a team plan interventions to ensure tiative, many organizations use the plan-do-study-act (PDSA)
that the improvement is implemented with the least amount quality improvement technique, also known as the plan-do-
of difficulty. Using force-field analysis, a team identifies check-act (PDCA) model. The PDSA model emphasizes
driving forces (those encouraging change) and restraining implementing initiatives on a small scale first and rolling out
forces (those discouraging change) for a planned action. The to a larger group as success is recognized.
78
Sidebar 4-4.
Case Study
One 400-bed hospital received 35% of its admissions from the emergency department (ED). Both public relations and
service quality problems were expressed by patients, family members, and physicians. Phase 1 of the investigational phase of
the improvement process used traditional performance improvement tools. The team members developed a flowchart to
better understand the situation and identified points in the process where patients might experience waits. They limited the
scope of the project to non-ambulance patients (ambulance patients went immediately to an ED bed without a wait). The
team identified the following four measures to track ED wait time:
1. Total patient wait time in ED (arrival to discharge, in minutes)
2. Type of caregiver involved
3. Amount of time care was delivered
4. Amount of delay patient experienced (all the time a patient was not receiving care, such as waiting for lab tests, physician
visits, or instructions)
Initial data showed frequent 3.5-hour waits (out of statistical control on a control chart), with an overall average wait time of
2 hours. A cause-and-effect diagram was created, and multivoting occurred to identify the three most likely causes, which
were (1) high patient arrival rates, (2) high patient acuity rates, and (3) clinic closures on weekends. Scatter diagrams were
created to detect correlations between these causes and actual wait times. No significant relationship was found for any of the
causes. The ED team kept diaries in an attempt to capture the special cause for the long wait times; no explanations were
revealed.
Phase 2 of the investigational phase used systems thinking tools. The team decided that traditional problem-solving tools
might not be appropriate if this was a system problem. A consultant in systems thinking was asked to study whether the long
waits were due to interdependencies in the ED processes. The ED staff helped to draw the complex causal loop cycles. A
computer simulation of the ED care system, using much of the data collected in Phase 1, plus department staffing patterns,
ED logbook data, and physical plant information about the ED, was added to the model.
Simulated improvements that worked in pilot testing included computerized ordering of medical records, creation of a
direct admissions process, alerts when system conditions predicted a rise in wait times, and zone assignments (staff members
assigned to specific sections of the ED).
PDSA is a robust performance improvement methodology • Identify measures that can be used to assess the success of the
that works particularly well in health care settings. It is strategy and whether the objective was reached.
designed to standardize processes and minimize variation, thus • Determine how to collect the measures of success.
eliminating mistakes and rework. By breaking function and • Involve the right people in the development and testing.
role into variables that can be measured, PDSA helps leaders • Resolve issues such as the following:
understand the clinical and medical environment and the – Who will be involved in the test?
method of providing care.5 – What must they know to participate in the test?
– What are the testing timetables?
PDSA involves identifying design or redesign opportunities, – How will the test be implemented?
setting priorities for improvement, and implementing the – Why is the idea being tested?
improvement project. Following are the components of the – What are the success factors?
PDSA cycle: – How will the process and outcomes of the test be measured
Plan and assessed?
• Develop or design a new process or redesign or improve an Do
existing process. • Run the test of the new or redesigned process, preferably on
• Determine how to test the new or redesigned process. a small scale.
79
• Collect data on the measures of success. group should be identified as responsible for monitoring the
Study data.
• Assess the results of the test.
• Determine whether the change was successful. For a changed process to become the standard, procedure
• Identify any lessons learned. manuals and policies must be updated with the change
Act approach. Continuing education and reinforcement of new
• Implement the change permanently. desired behaviors are also important. With support provided
OR by leaders, teams ensure that staff members are trained and
• Modify it and run it through another testing cycle. retrained as necessary about the process redesign, how to per-
OR form the steps of the new process, and the links between the
• Abandon it and develop a new approach to test. steps. Competence assessment must be regular and ongoing to
ensure that staff members maintain the skills necessary to fol-
A single initiative can involve a number of different testing low the redesigned process. Competence analysis of employees
phases or different change strategies and can therefore require must include evaluation of their implementation and mastery
the use of consecutive PDSA cycles. of the changed process.
There are many advantages to using a performance improvement See Sidebar 4-5, pages 81–82, for detailed information on Six
model such as PDSA to continuously evaluate improvement and Sigma, another performance improvement approach that
determine variation from the standard. PDSA provides a contin- organizations can use.
uous loop of quality monitoring based on data and allows for
objective examination of a process.5 It also outlines accountabili- Summary
ty for staff members and engages them in the performance Data are used in virtually all stages of the performance
improvement process. improvement process. They are used to identify improvement
opportunities and prioritize them in terms of their expected
Ensuring Ongoing Performance return for the organization. Data are also used to clarify
Measurement expected results of performance improvement efforts and to
Making an improvement in an organization requires a great deal specify outcomes that should be reached. Qualitative data are
of attention, time, and resources. Maintaining the improvement often used to determine specific performance improvement
also requires extensive attention. An organization should mea- team membership, and return-on-investment data are used to
sure the success of an initiative and verify that any performance determine the appropriate application of resources for the
improvements are maintained. This requires ongoing access to team. Finally, monitoring data are used to provide leaders with
information about how the redesigned process is operating. If oversight of the team, to ensure that improvements are imple-
data reveal any recurrence of the performance problem, this mented appropriately, and to see that gains become permanent
recurrence should be quickly addressed. and a stable part of organization performance.
Data that reveal expected outcomes should form the basis for Genuine performance improvement relies on a systematic
monitoring data and analysis. Run charts and control charts approach to the use of data in all phases of the process. Effective,
can be used to monitor performance improvement and to doc- accurate use of data forms the foundation for sustained, substan-
ument the results of the team’s efforts. A specific individual or tial improvement in organizational performance.
80
Sidebar 4-5.
Six Sigma
Six Sigma is one performance improvement approach that organizations can use to implement new processes or redesign
existing ones. Six Sigma is a set of practices originally developed by Motorola to systematically improve processes by eliminat-
ing defects.9 A defect is defined as a nonconformity of a product or service to its specifications. Six Sigma was heavily inspired
by six preceding decades of quality improvement methodologies such as quality control, total quality management (TQM),
and zero defects. Like its predecessors, Six Sigma asserts the following:
• Continuously reducing variation in process outputs is key to success.
• Processes can be measured, analyzed, improved, and controlled.
• Succeeding at achieving sustained quality improvement requires commitment from the entire organization, particularly
from top-level management.
The term Six Sigma refers to the ability of highly capable processes to produce output within specifications. In particular,
processes that operate with Six Sigma quality produce at defect levels below 3.4 defects per million opportunities (DPMO).10
Six Sigma’s implicit goal is to improve all processes to that level of quality or better.
The core of the Six Sigma methodology is a data-driven, systematic approach to problem solving, with a focus on customer
impact. Statistical tools and analysis are often useful in the process. However, it is a mistake to view the core of the Six Sigma
methodology as statistics; an acceptable Six Sigma project can be started with only rudimentary statistical tools.
Six Sigma has two key methodologies,11 both inspired by W. Edwards Deming’s plan-do-check-act cycle. The first method-
ology, define-measure-analyze-improve-control (DMAIC), is used to improve an existing business process. The steps involved
in DMAIC are as follows:
• Define the process improvement goals.
• Measure the current process and collect relevant data for future comparison.
• Analyze to verify the relationship and causality of factors. Determine what the relationship is and attempt to ensure that all
factors have been considered.
• Improve or optimize the process based on the analysis.
• Control to ensure that any variances are corrected before they result in defects.
The second methodology, design-measure-analyze-design-verify (DMADV), is used to create new process designs for pre-
dictable, defect-free performance. The steps involved in DMADV are as follows:
• Define the goals of the design activity.
• Measure and identify critical qualities, process capabilities, and risk assessments.
• Analyze to develop and design alternatives, create high-level design, and evaluate design capability to select the best design.
• Design details, optimize the design, and plan for design verification. This phase may require simulations.
• Verify the design, set up pilot runs, implement the process, and hand over the process to the process owners.
Six Sigma identifies several key roles for its successful implementation, as follows12:
• Executive leaders are responsible for setting up a vision for Six Sigma implementation. They also empower the other role
holders with the freedom and resources to explore new ideas for breakthrough improvements.
• Champions are responsible for implementing Six Sigma across the organization in an integrated manner.
(continued)
81

Six Sigma
• Master black belts, identified by champions, act as in-house expert coaches on Six Sigma for the organization. They devote
100% of their time to Six Sigma. They assist champions and guide black belts and green belts. Apart from dealing with the
usual rigor of statistics, they spend their time ensuring integrated deployment of Six Sigma across various functions and
departments.
• Black belts operate under master black belts to apply Six Sigma methodology to specific projects. They devote 100% of
their time to Six Sigma. They primarily focus on Six Sigma project execution, whereas champions and master black belts
focus on identifying projects/functions for Six Sigma.
• Green belts are employees who take up Six Sigma implementation along with their other job responsibilities. They operate
under the guidance of black belts and support them in achieving the overall results.
• Yellow belts are employees who have been trained in Six Sigma techniques as part of a corporatewide initiative but have not
completed a Six Sigma project and are not expected to actively engage in quality improvement activities.10
REFERENCES 7. Dlugacz Y., et al.: QM Environment: The Meaning of Measures Using

Quality Management Indicators to Improve Patient Care. Lake
1. Spath P.: Move for measurement to data intelligence. Hosp Peer Rev
Success, New York: North Shore–Long Island Jewish Health
pp. 174–176, Dec. 2004.
System, Center for Learning and Innovation, 2007.
2. Dlugacz Y.: The Quality Handbook for Health Care Organizations: A
8. Dlugacz Y.: Practical Applications of Quality Data. 2007 Joint
Manager’s Guide to Tools and Programs. San Francisco: Jossey-Bass,
Commission Resources Invitational Training Conference, Chicago,
2004.
Jan. 2–6, 2007.
3. Anthony M., Higgins P.: Maximizing the utility of
9. Motorola: FAQs: What Is Six Sigma? http://www.motorola.com/
interorganizational data using concept mapping. J Nurs Adm
content.jsp?globalObjectId=3088 (accessed Aug. 1, 2007).
5:233–240, May 2006.
10. Motorola: Six Sigma Dictionary. http://www.motorola.com/
4. Trochim W., Kane M.: Concept mapping: An introduction to
content.jsp?globalObjectId=3074-5804 (accessed Aug. 1, 2007).
structured conceptualization in health care. Int J Qual Health Care
11. De Feo J., Barnard W.: Juran Institute's Six Sigma Breakthrough and
3:187–191, May 2005.
Beyond: Quality Performance Breakthrough Methods. New York City:
5. Dlugacz Y.: Measuring Health Care: Using Quality Data for
McGraw-Hill, 2004.
Operational, Financial, and Clinical Improvement. San Francisco:
12. Harry M., Schroeder R.: Six Sigma: The Breakthrough Management
Jossey-Bass, 2006.
Strategy Revolutionizing the World's Top Corporations. New York City:
6. Joint Commission Resources (JCR).: Managing Performance
Random House, 2000.
Measurement Data in Health Care. Oakbrook Terrace, IL: JCR,
2001.
82
A Data-Driven Accreditation Process
Chapter 5
D ata management, performance measurement, and per-

formance improvement all play critical roles in The
Joint Commission’s accreditation process. Data are used to
Within the accreditation process, organizations are assessed
on their compliance with applicable standards, National
Patient Safety Goals, and other accreditation requirements.
help surveyors focus on areas within an organization that The accreditation process is made up of an unannounced
might represent important opportunities for improvement. survey and several presurvey components. The unannounced
Similarly, surveyors may use data to identify areas where survey is the on-site evaluation of an organization, and
organization performance is above average. Data also form presurvey components include the Periodic Performance
the focus of discussion between a surveyor and organization Review (PPR) and the Priority Focus Process (PFP).
staff members with respect to performance improvement
efforts and how the organization uses data to continuously This chapter provides an overview of the accreditation
improve its processes of care. process and highlights the role of data management and per-
formance improvement in that process. For a detailed expla-
Joint Commission accreditation and certification provide an nation, see the accreditation or certification manual for each
organization with the opportunity to participate in a thor- program.
ough and ongoing evaluation of processes and performance,
which is the cornerstone for continuous improvement. By Periodic Performance Review
achieving continuous compliance with Joint Commission The PPR is a compliance assessment tool designed to facili-
standards, an organization can achieve high-level perfor- tate an organization’s own continuous monitoring and per-
mance and can deliver safe, high-quality care to patients formance improvement. It helps an organization self-assess
while achieving a productive and satisfying work environ- compliance and puts the “continuous” in continuous
ment for the staff. compliance.
The Joint Commission’s accreditation process focuses on sys- The PPR facilitates an organization’s review of applicable
tems that are critical to patient safety and quality of care. standards, Accreditation Participation Requirements (APRs),
85
and National Patient Safety Goals. It requires organizations to review of an organization’s MOS and through observation dur-
assess compliance, identify areas of noncompliance, develop ing the on-site survey.
and implement plans of action, and identify measures by
which they can gauge success in carrying out those plans. By PPR options 2 and 3 use an on-site survey to evaluate compli-
participating in the PPR, an organization is better able to ance with all applicable standards, National Patient Safety Goals,
incorporate Joint Commission standards into routine opera- and other accreditation requirements. This survey uses the PFP
tions, which helps ensure the provision of safe, high-quality and tracer methodology (described later in this chapter, on pages
care on an ongoing basis. 87–89) to evaluate compliance. This on-site survey is typically
one-third the length of the organization’s last full survey.
An accredited organization* must complete a PPR (or one of
its three options) each year. Organizations that choose to com-
plete the full PPR or option 1 will use an online tool to evalu- TIP
ate compliance. If the organization identifies an area in need of Because organizations have continuous access to the
improvement, it must develop a Plan of Action and identify online PPR tool, they can use the tool throughout the
the measures that it will use to confirm that the problem has year to identify areas of noncompliance, complete
been resolved. A Plan of Action is a detailed description of how Plans of Action, and create Measures of Success. By
an organization plans to bring into compliance any standard continuously working on the tool, organizations do
identified as not compliant. Depending on what PPR option not have to rush to finish and can identify issues and
the organization selects, Joint Commission staff must approve implement solutions at an appropriate pace. By using
any plans of action, and surveyors will validate that measures the tool in this manner, organizations can ensure a
were implemented and effective at the next on-site survey. continuous approach to performance improvement.
When an organization is conducting a PPR, if noncompliance

with a standard is determined, elements of performance (EPs) in Priority Focus Process
some standards require that the organization identify a Measure The PFP guides the surveyor(s) in planning and conducting an
of Success (MOS) by which the success of any improvement organization’s on-site survey. The PFP uses an online tool to
plans can be evaluated. EPs that require this are identified in the transform data collected before an organization’s survey into
accreditation manuals and the automated PPR tool. information that customizes the survey process to the organi-
zation’s specific needs.
An MOS is a numeric or quantifiable measure, usually related to
an audit, that determines whether the action an organization has The PFP helps focus survey activities on the issues that are
taken has been effective and is sustained. For example, if an most relevant to patient safety and quality of care at the organ-
organization conducts an audit of patient records and finds that ization being surveyed. Information from a variety of
fewer than half of patients with pain had regular re-assessment sources—such as the organization’s electronic application for
and follow-up documented according to criteria developed by accreditation (e-App), previous survey findings, non-self-
the organization, the organization needs to determine what num- reported sentinel events, performance measurement data
ber of records to sample randomly on an ongoing basis to show (when applicable), complaints made to the Joint Commission’s
compliance levels after implementing improvement plans. Full Office of Quality Management, and publicly available external
compliance is 90% to 100%, partial compliance is 80% to 89%, data (such as MedPAR data for hospitals, Nursing Home
and noncompliance is less than 80%. Thus, an organization’s Compare for long term care and Medicare/Medicaid certifica-
MOS might read “90% of records of patients with pain have reg- tion-based long term care, Home Health Compare for home
ular re-assessment/follow-up documented, as defined by policy.” care, and lab proficiency test failure for laboratories)—are inte-
An organization may make the requirement more stringent. An grated to identify the important clinical/service groups (CSGs)
organization should also state how often it plans to measure com- and priority focus areas (PFAs) for that organization. PFP data
pliance and what the sample size should be. The surveyor will are program specific. Note: Self-reported sentinel events and
confirm the implementation of any Plans of Action through a PPR data are not included in the PFP.
* Note: This requirement does not apply to critical access hospitals or office-based surgery practices.
86
Chapter 5: A Data-Driven Accreditation Process
CSGs define patient and/or service populations for which data vided at all times and that organizations should not have to
can be collected; each accreditation program has a list of CSGs “ramp up” for an on-site survey. Staff members focus on their
relevant to the patient populations it serves and the services it everyday processes rather than on making “quick fixes” to com-
provides. PFAs are processes, systems, and structures in a ply with standards just before a survey, and surveyors are able to
health care organization that significantly affect safety and/or observe care being provided under typical circumstances.
the quality of care provided. The PFAs are as follows:
• Assessment and care/services During an on-site survey, the Joint Commission evaluates an
• Communication organization’s performance of functions and processes aimed at
• Credentialed practitioners continuously improving patient outcomes. This assessment is
• Equipment use accomplished by evaluating an organization’s compliance with
• Infection control applicable standards based on the following:
• Information management • Tracers of the care delivered to patients
• Medication management • Verbal and written information provided to the Joint
• Organizational structure Commission
• Orientation and training • On-site observations and interviews by Joint Commission
• Patient safety surveyors
• Physical environment • Documents provided by the organization
• Quality improvement expertise and activity
• Rights and ethics Tracer Methodology
• Staffing The tracer methodology is the cornerstone of the Joint
• Analytical procedures* Commission’s on-site survey. It is used as a way to evaluate an
organization’s compliance with standards and requirements.
A summary report listing an organization’s top four or five Outside formal conferences/interviews, much of a survey con-
CSGs and PFAs is made available to an organization on its sists of reviewing issues in the form of the tracer methodology.
secure The Joint Commission ConnectTM extranet site. These The tracer methodology helps surveyors assess the care, treat-
data are updated quarterly, as changes warrant. ment, and services provided by an organization by following
the actual care experiences of patients within the different areas
In addition, prior to survey, surveyors access an organization’s of a health care organization.
PFA and CSG information and plan initial tracer activities
based on this information. (Tracers are discussed in more detail Through the tracer methodology, surveyors can identify per-
later in this chapter; see “Tracer Methodology.”) Surveyors formance issues in one or more steps of a process or in the
choose patients to trace based on the identified CSGs. As a interfaces between processes. There are two types of tracers: (1)
survey progresses, a surveyor may find other priority areas that individual tracers and (2) individual-based system tracers. The
need to be addressed and may begin to focus less on the PFP– primary difference between the two types is that system tracers
suggested list and more on what he or she is finding. focus on important organizationwide processes or functions,
and individual tracers address the experience of a patient
See Sidebar 5-1 on page 88 for additional information on PFP receiving care or services.
data. For information on the Joint Commission’s Strategic
Surveillance System, see Sidebar 5-2 on page 88. Individual Tracers
Individual tracers “trace” the care experiences that an individ-
The On-Site Survey ual has while at an organization. During an individual tracer,
A critical component of the accreditation process is the on-site the surveyor uses the organization’s PFP information to iden-
survey, which occurs approximately every three years (biannual- tify specific patients and to follow the care of those patients
ly for laboratory programs) and is conducted on an unan- from admission through discharge or transfer. (For laborato-
nounced basis. The unannounced aspect of the on-site survey ries, patient samples are traced from physician order through
emphasizes the tenet that safe, high-quality care should be pro- notification and documentation.) Surveyors try to select
* Note: This PFA is applicable to the laboratory program only.
87
Sidebar 5-1.
Responding to Quarterly Priority Focus Process Data
As previously mentioned, The Joint Commission offers Priority Focus Process (PFP) reports on a quarterly basis via an orga-
nization’s extranet site. An organization receives an updated report only if there are changes to its PFAs or clinical/service
groups (CSGs). The Joint Commission does not require a specific response to a quarterly PFP report. However, any changes
in an organization’s PFAs or CSGs could serve as a warning sign for a potential issue within the organization. Organizations
should consider examining the systems and processes associated with any PFAs or CSGs that are newly identified in their
quarterly report. For example, organizations may want to use these PFAs or CSGs when selecting patients and processes for
mock tracers (see page 89).
Organizations may want to use the PFAs or CSGs as sources for proactive risk assessment. By using tools such as failure
mode and effects analysis and root cause analysis, organizations can identify system problems associated with these areas or
groups and develop solutions. Such a proactive approach can help an organization work toward continuing compliance with
the standards and the safest, highest-quality care.
Sidebar 5-2.
The Strategic Surveillance System
In July 2007, The Joint Commission released the Strategic Surveillance System (S3), which is a series of risk assessment and
performance measure comparison reports for hospitals. This tool is designed to help hospitals improve their care processes and
prioritize actions to take for improvement.
The idea is that S3 will be a suite of information tools. Currently there is one defined tool within S3 called Performance
Risk Assessment (PRA). By using this tool, an organization can analyze its comparative performance against national, state,
and other benchmarks and trend historical data to identify risks, set priorities for attention, and undertake appropriate
improvements.
S3 uses data the Joint Commission currently has, including past survey findings, ORYX core measure data, data from
the Office of Quality Monitoring, electronic application for accreditation (e-App) data, and MedPAR data. Reports are post-
ed and updated quarterly on each organization’s secure extranet site and are available to the individuals identified in an
organization’s e-App.
S3 helps drive and monitor systems improvement throughout an organization by providing ongoing quantitative feedback.
Hospitals with a common owner have the ability to compare S3 data among themselves to identify trends or common areas
for improvement. For more information about S3, see http://www.jointcommission.org.
patients who are in the organization’s top CSGs, who cross When conducting an individual tracer, the surveyor(s) may start
programs and/or have received care in multiple areas (such as the process where the patient is currently located. The surveyor(s)
a nursing home resident who is admitted to a hospital and can then move to where the patient first entered the organiza-
then transferred to a rehabilitation facility), and who have been tion’s systems, to an area of care provided to the patient that may
in contact with areas related to the individual-based system be a priority for that organization, or to any areas in which the
topics (that is, infection control, medication management, and patient received care, treatment, and services. The order will vary.
data use). Viewing an organization’s systems from the patient’s Along the way, the surveyor(s) will speak with health care staff
perspective allows an evaluation of both the components of a members who actually provided care to that individual tracer
system and how the different systems work together as a patient; if a particular staff member is not available, the survey-
whole. or(s) will speak with another staff member who provides the
same type of care.
88
Chapter 5: A Data-Driven Accreditation Process
During an individual tracer, the surveyor(s) will not only exam- As part of a data-use tracer, a surveyor will discuss with orga-
ine the individual components of a system but will also evaluate nization staff members the strengths and weaknesses of the
how the components of a system interact with each other. In organization’s information management process for meeting
other words, the surveyor(s) will look at the care, treatment, and internal and external information needs and the organization’s
services provided by each department/unit/program and service methods of data analysis, interpretation, and use. The surveyor
as well as how departments/units/programs and services work will also examine how the organization uses resulting data and
together. Topics of discussion are guided by PFP information and information.
previous survey findings. For example, if an organization’s PFP
report identifies communication and information management Conducting Mock Tracers
as two of its PFAs and general surgery as one of its top CSGs, the An organization can perform its own patient tracers, called
surveyor is likely to choose one or more patients who underwent “mock tracers,” to obtain a complete picture of its perfor-
surgery to be subjects of individual tracers. While following the mance. Doing so on an unannounced basis allows an organi-
route the individual took through the various areas of the hospi- zation to gain more accurate impressions of how well it is
tal, the surveyor might talk to staff about issues such as processes delivering safe, high-quality care, treatment, and services on an
for communicating to all clinical staff any allergies and current ongoing basis. This exercise helps generate ideas for ways to
prescriptions or ensuring that surgery is performed on the correct improve systems and processes within an organization and
site. helps staff members become more familiar with the tracer
methodology. Mock tracers should involve many different dis-
No two tracers are the same. It is not possible to predict how a ciplines, including nurses, physicians, social workers, techni-
tracer will occur and which processes and subprocesses will be cians, and housekeeping staff members.
examined. Each tracer is conducted in a time frame ranging
from 90 minutes or less to 3 hours. Extenuating circumstances, Evidence of Standards Compliance
such as complexity, may lengthen or shorten an individual trac- Following the evaluation of an organization’s performance of
er. Although the activities may be followed sequentially for the functions and processes during the on-site survey, a surveyor
tracer selected, there is no mandated order for the visits to vari- reviews the results of his or her findings and the findings of any
ous units or departments throughout an organization. other surveyors on the survey team and, with the use of laptop-
based decision support software, produces the organization’s
Individual-Based System Tracers Accreditation Survey Findings Report. This report includes, as
An individual-based system tracer explores one specific system appropriate, requirements for improvement and any supple-
across an organization. During this activity, the surveyor(s) eval- mental findings.
uates the system, including the integration of related processes
and the coordination and communication among disciplines and For every Requirement for Improvement cited in an organiza-
departments in those processes. The surveyor uses information tion’s report, the organization must submit an Evidence of
obtained from individual tracers and discussions with staff mem- Standards Compliance (ESC) report within 45 days of survey.
bers to trace one or more of the following “systems”: This report details the actions the organization took to bring
• Data use, which looks at how the organization collects, ana- itself into compliance with a standard or clarifies why the
lyzes, and interprets data to improve patient safety and care organization believes it was in compliance with the standard at
• Infection control, which focuses on the organization’s process- the time of survey. An ESC may include a MOS.
es for the prevention, control, and surveillance of infection
• Medication management, which explores the organization’s As with the MOS in a Plan of Action for the PPR, an organi-
medication processes and subprocesses from procurement to zation needs to decide which measurable, objective criteria can
monitoring of effects, as well as potential risk points show that the EP is in compliance. Sources for criteria could
include medical records, staff interviews, facility inspections,
Because errors are so prevalent when patients are handed off or performance improvement data. It is helpful to use the same
between programs, units, organizations, and individual practi- type of data that was originally used to show that the EP was
tioners, surveyors pay particular attention to how staff mem- not in compliance. This enables the organization to show
bers and processes interact and how care is coordinated. improvement and verify a track record of accomplishment.
89
Summary
Striving toward continuous compliance and monitoring efforts
to attain continuous compliance provides a framework for an
organization’s performance improvement strategy. Joint
Commission standards provide a holistic look at organization-
al performance, and following these standards and require-
ments can help organizations achieve high-quality care and
streamlined performance. Just as data management is critical
to Joint Commission accreditation, it is also critical to an orga-
nization’s success in achieving accreditation and providing the
safest and most efficient high-quality care.
90
Case Studies—A Spotlight on Success
Chapter 6
T he previous chapters discuss the process of data management, give suggestions on data collection and analysis, and take a
close look at how to use data to improve performance. This chapter provides five real-world examples of how organiza-
tions throughout the country have used data to improve performance, increase patient safety, and affect quality of care.
Although the information in these examples may not be directly applicable to your health care organization, the approaches,
processes, and conceptual framework these organizations used can help any organization design and develop performance
improvement (PI) programs that are rooted in reality and are practical for achieving and maintaining success.
93
CASE STUDY 1:
NOVANT HEALTH
Several years ago, Novant Health, Inc.—a large, integrated health Forming a Team
system located in Winston-Salem/Charlotte, North Carolina— To respond to the issues it discovered, Novant created an inter-
participated in a 13-organization collaborative, sponsored by the disciplinary, corporatewide PI team. Novant Health has eight
Veterans Health Administration (VHA), to determine where hospitals, two nursing home and senior residential facilities, and
adverse drug events (ADEs) were occurring in hospitals. Within multiple physician practices, outpatient surgery centers, and
this collaborative, ADEs associated with warfarin—an anticoag- rehabilitation and community health outreach programs. The
ulation drug often used in treating blood clots—were some of the organization is large, and it was essential that all departments
most common medication events noted and the most common involved in the care of patients provide ideas for improvements,
within the Novant system. identify barriers, provide feedback, and help monitor the data.
Ultimately, the PI team included three vice presidents of medical
About the same time, Novant was establishing its patient safe- affairs, four pharmacists, two pharmacy directors, three nursing
ty program and determining PI goals that would help improve leaders, two outpatient medical directors, a senior vice president
patient safety and quality of care throughout the organization. of clinical improvement, a medical director of clinical improve-
These goals were tied to leadership compensation and repre- ment, and clinical improvement department support personnel.
sented an organizationwide effort to improve performance. The team was supported by a physician consultant who had been
Based on the results of the ADE collaborative, the executive involved in the VHA collaborative. “The team had broad repre-
leadership of Novant determined that the first of its long-term sentation of content experts and frontline personnel, as well as
PI goals, to be measured over three years, would be to reduce leadership and support personnel,” says Lederer. “When con-
warfarin-related errors. ducting any performance improvement project, you must have
the input and collaboration of content experts and frontline staff
Determining the Nature of the Problem members, but also the support and input of leadership. Without
To determine what factors were contributing to warfarin-asso- the three-year PI goal set by executive leadership, the focus and
ciated errors, Novant took a close look at the ADE collabora- importance of the project may have been lost due to the complex
tive data. The data showed that warfarin-related incidents fell nature of our organization and the difficulty in aiming and mov-
into two distinct categories: those related to inpatients and ing together to achieve a goal.”
those related to outpatients. The inpatient errors tended to
occur in patients with many comorbidities who were taking Testing Rapid Change
lots of other medications. The other medications sometimes The Novant PI team examined possible solutions and began run-
interacted with warfarin, with problematic results. “Warfarin ning mini plan-do-study-act (PDSA) cycles (see Chapter 4).
has a low therapeutic-to-toxic ratio,” says James Lederer, “These cycles allowed us to test different ideas on a small scale
M.D., medical director of clinical improvement for Novant and determine what would work best,” says Lederer. “For exam-
Health. “In other words, a certain dose of warfarin can be ther- ple, we tested potential interventions with 1 or 2 volunteer doc-
apeutic, but a slight increase in dose can be toxic. In patients tors instead of 100 doctors because it’s a lot easier to adjust an
with multiple comorbidities who are taking many different intervention on a small scale than across an organization.”
medications, the changes between therapeutic and toxic can
happen quickly.” Vigilant patient monitoring is necessary to Using Effective Interventions
combat this potential problem. Within the inpatient environment, Novant ultimately developed
a multifaceted approach to reducing error. First, the PI team
Within the outpatient setting, problems existed because physi- established a critical value for anticoagulation. This critical
cians did not have a standard approach for dosing warfarin and value—an International Normalized Ratio (INR) of 3.0 or
monitoring and responding to changes in patient response. more—was determined from the PDSA cycles and literature on
Patients on warfarin were not followed consistently after they the topic. If a patient’s anticoagulation level reaches this critical
left the outpatient setting. value, the nurse is required to place a direct call to the responsi-
ble physician, alert him or her about the elevated INR level, and
94
Chapter 6: Case Studies—A Spotlight on Success
Figure 6-1. Warfarin Index Sheet

This index sheet shows the categories into which pharmacists grouped patients with high International Normalized Ratio (INR)
values.
Category Definition Example
No adverse No relationship between trigger and drug • High INR due to liver disease.
drug event therapy. • Rising INR, routine dose adjustment made promptly. No opportunity
(ADE) to respond earlier.
Trigger related to drug therapy, but therapy • Loading dose appropriate or drug interactions noted for dosage
was appropriate and/or adjusted promptly. adjustment but still resulted in trigger event. There was no
opportunity to act sooner or to have chosen a better regimen.
B DO NOT USE Was a medication error that did not reach the patient—not applicable to
ADE triggering.
C No harm, routine adjustments made, but Distinguished from “A” in that an opportunity to act sooner within the
minor deviation from standard or “best facility existed or therapeutic regimen may have been better selected.
practice” therapy noted. • Loading dose might have been slightly lower, resulting in not
overshooting the trigger. Dosage adjustment could have been made
sooner.
D No harm, but more intense monitoring, Distinguished from “C” in that changes in drug therapy or an abrupt
treatment, or abrupt change in drug therapy intervention rather than routine “tweaking” of dosage or more intense
required to avoid harm. Represents a monitoring or treatment is required to avoid harm.
significant deviation from “best practice” • Significantly high loading dose or missed drug interaction, resulting in
therapy. trigger event, and abrupt discontinuation of warfarin or more frequent
monitoring of INR. No harm associated.
E Temporary symptomatic harm, treatment, or Distinguished from “D” in that symptomatic harm exists. Note that a high
abrupt change in drug therapy required. lab trigger value in absence of physical symptoms is not considered to
be “harm.”
• Bruising or bleeding from anticoagulation, treated with vitamin K.
F Temporary symptomatic harm AND required Temporary symptomatic harm, AND
initial hospitalization, prolonged length of stay • Event occurred prior to admission and patient admitted or presented
(LOS), movement to higher level of care. to ED because of the event (for outpatients), or
• LOS prolonged, or
• Patient moved from floor to ICU.
G Contributed to or resulted in permanent Loss of limb or organ, deafness.
patient harm.
H Intervention required to sustain life. “Code blue” called, and patient survived.
I Contributed to the patient’s death. Any event contributing to the patient’s death.
Source: Novant Health, Winston-Salem/Charlotte, NC. Used with permission
await orders to continue with current dosing or wait for further Within the outpatient environment, the organization looked
dosing orders. To ensure proper monitoring, the organization at best practices surrounding the use of warfarin and deter-
requires that the anticoagulation levels of any inpatient on anti- mined a common practice to be used by all physicians. In addi-
coagulation medication be checked at least every 72 hours. tion, the organization provided education to all outpatient
physicians on the dosing and risks of warfarin. The organiza-
In addition to establishing a critical anticoagulation value and tion also developed an automated system to help ensure that
ensuring proper monitoring, the organization also implement- nurses regularly followed up with patients who were on war-
ed a program to educate physicians who worked in the inpa- farin to determine whether any issues were present.
tient environment on effective warfarin dosing and the risks
associated with warfarin.
95
Measuring Success ership meetings, and newsletters. The organization also used a
To monitor the effects of its warfarin interventions, designat- dashboard (see Chapter 3) to illustrate success and areas for
ed Novant pharmacists reviewed inpatient charts for seven improvement. The PI team received data feedback monthly,
working days every month, looking for all patients who had and the leadership teams received reports quarterly. The orga-
critical INR values greater than 3.0 or orders for anticoagula- nization as a whole received updates every four to six months.
tion-reversal drugs, including vitamin K and fresh frozen plas- The exception was for individual nurses or physicians who
ma. When an event was discovered in a patient’s chart, the were involved in significant-harm events; these events were
pharmacist would classify it using a predetermined index (see addressed as they occurred so that timely feedback and educa-
Figure 6-1 on page 95). This allowed the organization to deter- tion could take place.
mine when an elevated INR was due to a warfarin incident or
an issue not related to the use of warfarin, such as the presence Novant also communicated its results to the community. “It is
of liver disease. a good idea to proactively tell the story of improvement to the
community. It not only makes patients more confident in the
To ensure consistency in data analysis, all pharmacists who organization but helps set expectations for patients and
classified data received the same training given by the physi- empowers them to participate in their own care,” says Lederer.
cian consultant. Dr. Lederer also reviewed all patient charts to “The more involved patients are in an initiative, the more suc-
ensure proper classification and provided instructional feed- cessful it is going to be.”
back in cases where he determined a different harm category
for a given patient. Results of the Initiative
The results of Novant’s warfarin initiative were very positive.
The data were standardized to 1,000 patient days. This Inpatient warfarin incidents dropped by 38%, and outpatient
allowed the organization to normalize big and small hospitals. incidents dropped by 51%. Recently, the organization devel-
“These events, while serious, are extremely rare, and in a small oped a computer system to help catch warfarin incidents even
hospital, they may occur very infrequently. By standardizing sooner. As a patient’s warfarin level rises toward the critical
the results of all hospitals in our system to 1,000 patient days, value, the computer system notifies the pharmacist, who com-
we were able to emphasize the importance of the problem and municates with the nurse that the patient should be monitored
make it possible to compare rates across hospitals,” says more closely. “This allows us to catch problems more upstream
Lederer. and prevent issues before they become problems,” says Lederer.
Data were shared with the PI team, organization leadership, By collecting, analyzing, and responding to data, Novant
medical staff (including both inpatient and outpatient physi- Health was able to reduce its ADEs related to warfarin in both
cians), and frontline staff members. Information was shared the inpatient and outpatient settings. The organization contin-
through Microsoft® PowerPoint® presentations, memos, lead- ues to use the interventions implemented in this project.
96
CASE STUDY 2:
GREATER CINCINNATI PATIENT SAFETY ICU COLLABORATIVE
When a local Department of Veterans Affairs (VA) investigator
and the president of the Greater Cincinnati Health Council Sidebar 6-1.
teamed up and submitted a proposal in 2003, 10 hospitals in the Greater Cincinnati Patient Safety ICU
greater Cincinnati area agreed to participate in a two-year collab- Collaborative: Participating Hospitals*
orative to reduce central line infections in the intensive care unit
(ICU) and surgical site infections in the operating room (OR). The following 10 hospitals participated in the
(See Sidebar 6-1, at right, for a list of the participating hospitals.) collaborative:
The project focused on these areas because recommendations to 1. Bethesda North Hospital
reduce nosocomial infections related to the ICU and OR are well 2. Cincinnati Children’s Hospital Medical Center
studied and largely accepted, and substantial variability exists in 3. Cincinnati Department of Veterans Affairs Medical
the adoption of most of these recommendations. In addition, Center
improvement in these areas can have a significant impact on 4. Good Samaritan Hospital
patient safety and economic outcomes for a health care organiza- 5. Mercy Mount Airy
tion. For example, catheter-related bloodstream infections affect 6. St. Elizabeth Medical Center
hundreds of thousands of patients per year and can cost between 7. The Christ Hospital
$6,000 and $40,000 per episode. 8. The Fort Hamilton Hospital
9. The Jewish Hospital
Although many of the hospitals in the collaborative were 10.The University Hospital
already working on decreasing nosocomial infections, and
* ICU, intensive care unit.
almost all were collecting infection rate data, none of the hos-
pitals measured adherence to evidence-based practices. The
goals of the collaborative were to not only reduce central line Series concept and then in small groups critiqued implementa-
and surgical site infections but also to teach the rapid action tion plans that the project teams presented for achieving rapid-
cycle process of improvement to increase the use of evidence- cycle change. This intensive training was followed by alternating
based practices and develop an evidence-based practice monthly work-learning and reporting cycles, which is similar to
resource for the entire Cincinnati area. IHI’s Breakthrough Series process.
Developing a Team The Nature of Interventions

The CEO of each hospital participating in the collaborative To reduce central line infections, the collaborative selected
selected team members from multiple disciplines to spearhead the bundle practices associated with central line insertion, includ-
PI efforts. Typically, the hospital’s infection control practitioner ing the following:
led the team, which included the nurse manager of the interven- • Use chlorhexidine instead of betadine as a skin disinfectant.
tion unit (ICU or OR), nurse managers of related units, staff • Ensure maximum sterile barriers, including using a full
nurses, a nurse educator, pharmacy staff members, an intravenous patient drape during insertion and having the physician
(IV) therapist, the supply manager, and a physician champion. wear a sterile gown, gloves, mask, and cap.
Before beginning work, all the improvement teams met for a The collaborative identified multiple interventions for reducing
two-day kick-off learning session. One of the hospitals partici- surgical site infections, including focusing on the correct timing
pating in the collaborative—Cincinnati Children’s Hospital of preoperative antibiotics, reducing hypothermia, using clippers
Medical Center (CCHMC)—is an established, strong quality rather than razors for skin preparation, and glycemic control. Of
improvement–focused organization whose staff members had these, only improvement in timing of antibiotics was improved,
participated in the Institute for Healthcare Improvement’s with a goal that all preoperative antibiotics should be given with-
(IHI’s) Breakthrough Series training. During the kick-off meet- in 60 minutes of incision—except for some drugs, such as van-
ing, staff members from CCHMC presented the Breakthrough comycin, which should be given within 120 minutes of incision.
97
“We opted to pursue these particular interventions because To help staff members follow evidence-based practices regard-
they were widely supported in the literature, and we believed ing central line insertions, the collaborative developed a check-
they were the easiest and least expensive to implement. We list (see Figure 6-3 on page 100). Each hospital could add to
wanted to see if these interventions would be effective before the required information necessary for consistency in data col-
we looked at the more expensive, harder-to-implement lection across the collaborative. An ICU nurse was responsible
processes,” says Suzanne Brungs, project coordinator for the for filling out the checklist upon insertion of the line.
10-hospital collaborative. “Typically during central line insertion, an ICU nurse was usu-
ally present, so filling out the checklist was not a burden to the
Using Rapid Tests of Change staff, since they were in the room anyway,” says Brungs. To
Each hospital customized the previously mentioned interven- measure adherence with evidence-based practices, the collabo-
tions for the needs of its particular organization. A key part of rative looked at how often physicians and nurses adhered to
the implementation process was learning how to do a rapid test the practices on the checklist as compared with the number of
of change. “By making changes rapidly on a small scale, you checklists completed. All data related to central line insertions
can see immediately if something is going to work,” says were entered into a Web-based database.
Brungs. “For example, one hospital chose to work with a sin-
gle particular resident and nurse first to determine the best way In regard to surgical site infections, the participating organiza-
to use chlorhexidine and a full body drape. They tested sever- tions collected data on the number of surgical site infections as
al different drapes until they found a suitable one. Once the the numerator and the total number of surgical procedures,
resident and nurse found the appropriate approach, the orga- including vascular, neurological, orthopedic, obstetrics-gyne-
nization moved the plan out to the entire day shift and later to cology (OB-GYN), and cardiovascular procedures, as the
all shifts.” denominator. The collaborative also collected information on
how often antibiotics were given within the appropriate time
Another hospital created a central line cart that was brought to frame. Because data were sometimes hand collected, some hos-
the bedside any time a line was inserted. The project team at pitals created an “antibiotic before cut” (ABC) form on which
that hospital learned from other hospitals in the collaborative the staff could manually document the time that antibiotics
what was needed on the cart and then tested the cart and its were given and the time of cut. Some sites added these data
contents for several insertions with a few nurses, who provid- points to the bottom of an existing OR information sheet. At
ed feedback. Contents and the process for use of the cart were some sites, these data were collected electronically. PI team
refined with the nurses’ suggestions and then finalized for use leaders used electronically collected information to calculate
with all bedside insertions. the rate of adherence to the evidence-based practice.
Collecting Data Reporting Data

The 10 organizations were randomized to participate first in Project leaders for both the ICU and OR initiatives had weekly
either the ICU or the OR intervention, and nonparticipating phone calls with Brungs and met monthly to present their results
units served as the control for the units implementing change. to each other. They used PowerPoint presentations to show their
For example, the ICUs at the hospitals performing the OR ini- organization’s infection rates and compliance with evidence-
tiative formed the control for the ICUs performing the ICU based practice. “These meetings allowed a little friendly compe-
initiative, and vice versa. tition between hospitals,” says Brungs. “Those organizations that
were struggling obtained ideas about new strategies to try and
Each organization collected outcome (infection rate) and were motivated by the success of other facilities. They could go
process (adherence to evidence-based practices) data. For cen- back to their leadership and discuss the need for improvement.”
tral line infections, the organizations collected data on the
number of infections per the number of central line days. “We The teams also shared the performance results with frontline
standardized the data with a denominator of 1,000 central line staff members, leadership, and external audiences, both for-
days, so that organizations could compare to other hospitals in mally and informally. Leadership of each organization received
the project and to national data. In addition, we established quarterly reports comparing their organization’s results to the
consistent definitions for data collection to ensure that all data average and “best” in the project. The organizations used the
were comparable,” says Brungs. (See Figure 6-2 on page 99.) aforementioned PowerPoint presentations, as well as verbal
98
Figure 6-2. Measurement Information Form

This form shows how data definitions and collection methods were kept consistent.
Central Line–Associated Primary Bloodstream Infection Rate per

1,000 Central Line Days
Calculation Details
Numerator Definition: Number of central line–associated primary bloodstream infections (BSIs) in ICU patients with a
laboratory-confirmed BSI who had a central line in place within the 48-hour period before the development of the BSI, by
unit of attribution.
Denominator Definition: Number of central line days, for patients who have a central line in place and are receiving
care in the ICU. A patient with more than one central line should be counted as 1 central line day.
Definition of Terms:
• Use CDC guidelines (from Appendix A of CDC Guideline, MMWR Aug 9, 2002/51 (RR 10); 27–28)
Measurement Period: Measure monthly.
Calculate as: Number of central line–associated bloodstream infections/Number of central line days (x 1,000).
For example, if in February there were 12 cases of CR-BSIs, the number of cases would be 12 for that month. We want
to be able to understand that number as a proportion of the total number of days that patients had central lines. Thus, if
50 patients had central lines during the month and each, for purposes of example, kept their line for 5 days, the number
of catheter days would be 50 x 5 = 250 for February. The CR-BSI Rate per 1,000 catheter days then would be (2 / 250)
x 1,000 = 8 / 1,000 central line days.
Source: Greater Cincinnati Health Council. Used with permission. ICU, intensive care unit; CDC; Centers for Disease Control and Prevention; CR-BSI,
Catheter-Related Blood Stream Infection.
presentations, posters, hospital newsletters, and other means, Achieving Success

to communicate the data and the results of the collaborative. After two years, teams in the collaborative had reduced central
line infections by 50% while increasing adherence to evidence-
About the Greater Cincinnati Patient Safety ICU based practices, such as using chlorhexidine and the full body
Collaborative drape, from 30% to 95%. Surgical site infections were also
The Greater Cincinnati Patient Safety ICU Collaborative is part reduced. In addition, the organizations created a new model
of the Greater Cincinnati Health Council, a not-for-profit for facilitating improvement as a community, by sharing suc-
member service organization, which is dedicated to working cessful strategies, reducing rework and redundancies across
cooperatively with hospitals and other health care providers in sites, and speeding up the implementation process.
southwestern Ohio, northern Kentucky, and southeastern
Indiana to promote high-quality, cost-effective patient care. The The collaborative also achieved its goal of developing evidence-
council provides its members with services that enhance their based resources. All forms, data collection tools, results, and so
ability to deliver high-quality, cost-effective health care and well- forth are posted on the Greater Cincinnati Health Council’s Web
ness services to the people of the Tristate area, speaks for its site, at http://www.gchc.org. The Health Council involves 35 to
members to key audiences, and, on behalf of its membership, 40 health care organizations and serves as a resource for health
collaborates with community organizations in activities that care improvement in the greater Cincinnati area.
promote positive health outcomes.
99
Figure 6-3. Central Line Insertion Checklist

This form is an example of the type of checklist used to monitor the use of evidence-based practices in central line insertions.
FILL OUT FOR ALL LINE INSERTIONS
Insertion Date: ____________ Time: ______________
Type of line: ______________ Site: _______________
Guide wire change _____ Yes _____ No
Non-urgent ______ Urgent/emergency _____
OBSERVATIONS R-1 R-2 R-3 ATTENDING PICC RN
Wash/disinfect hands YES NO YES NO YES NO YES NO YES NO
Head cover YES NO YES NO YES NO YES NO YES NO
Mask YES NO YES NO YES NO YES NO YES NO
Chloraprep swab YES NO YES NO YES NO YES NO YES NO
Sterile surgeon gown YES NO YES NO YES NO YES NO YES NO
Sterile gloves YES NO YES NO YES NO YES NO YES NO
Full body drape YES NO YES NO YES NO YES NO YES NO
Sona site/site rite YES NO YES NO YES NO YES NO YES NO
Probe cover YES NO YES NO YES NO YES NO YES NO
Hospital admission date _____________ ICU admission date: __________

Hospital discharge date _____________ ICU discharge date: __________
Line removal date __________ Infection _____ No _____ Yes
Type: ______ CR-BSI ____ Lab confirmed bacteremia ______ Exit site infection ____ Colonized cath tip
Source: Greater Cincinnati Health Council. Used with permission.
100
CASE STUDY 3:
KINGS HARBOR MULTICARE CENTER
Kings Harbor Multicare Center is a 720-bed long term care very positive feedback about the change, and this led to a recom-
facility located in Bronx, New York. It serves a variety of dif- mendation to purchase an additional van and hire an additional
ferent patient populations, including skilled long term care driver. Together the vans would be used to transport families and
residents with medical needs, long-term psychiatric and friends to visit residents, transport outpatient rehabilitation
dementia residents, and subacute care patients; in addition, it patients to and from the organization, and transport residents to
provides outpatient rehabilitation. For many years, the loca- a variety of outings, including trips to the zoo, baseball games,
tion of the three-building campus presented some challenges the botanical gardens, museums, movies, and the library. The
for the organization. Access to public transportation, such as administrative secretary of the organization served as the sched-
bus and train lines, was difficult, and many of the residents uler and dispatcher and worked with residents, families, outpa-
and their families did not own cars. As a result, many of the tients, and drivers to coordinate schedules and ensure the safe
organization’s residents were isolated from friends and family transportation of people who needed it. The organization also
and received very few visitors. addressed potential communication issues by providing drivers
with walkie-talkies, so the drivers could easily communicate with
Leaders of the organization received feedback from staff mem- the administrative secretary if there was a problem.
bers, residents, and family members that transportation was a
significant issue for the organization. Residents were feeling Collecting Data
isolated, and organization leaders hypothesized that this was To measure the impact of the vans, Kings Harbor developed
having a negative effect on the well-being and quality of life of both a customer satisfaction survey and a resident satisfaction
some residents. In addition, hospitals that referred residents to survey. (See Figures 6-4 and 6-5 on page 102.) The customer
Kings Harbor indicated that most people value ease of trans- satisfaction survey was given to families and friends of resi-
portation when selecting a long term care facility and noted dents who used the van as well as to any outpatients who used
that Kings Harbor’s transportation issue was a potential con- the service. The resident satisfaction survey was given to resi-
cern to admissions. dents who used the vans and residents whose families used the
vans. The organization also tallied the number of resident trips
Because of all the feedback on transportation, Kings Harbor and family and outpatient visits, and it documented any spe-
decided to take action and address the issue. The organization cific customer or resident comments.
formed a multidisciplinary PI team that included representa-
tives from nursing, social services, rehabilitation, PI, admis- The PI committee met monthly to review the data. Data were
sions, recreation, medical staff, finance, and administration. shared with the organization’s leadership on a quarterly basis.
“We wanted to get everyone’s input on this problem,” says Kings Harbor used histograms and bar charts to display the
Hilary Rizzo, assistant administrator for Kings Harbor, data and also documented actual customer comments.
“because it affected many aspects of the organization. For
example, the admissions department is often the first place Results of the Program
where residents and their families express concerns about After adding one van, Kings Harbor saw a significant increase in
transportation. By having the admissions department be a part both customer and resident satisfaction as well as an increase in
of the performance improvement team, we were able to cap- the number of family visits and resident outings. The decision
ture this valuable perspective.” was made to add a third van and driver. Again, customer and res-
ident satisfaction increased. Ultimately, the organization pur-
Initially, the group looked at the activities and availability of the chased one more van and hired another driver. As a result of the
one van and driver the organization had at the time. The driver program, the number of family transportation episodes increased
and van were used predominantly for facility errands and occa- 46%, and the number of outpatient rehabilitation uses increased
sional resident trips. The group committed to a change in driver by 38%. Resident satisfaction with the transportation program
and van activity and had the driver start transporting families and hovered at 90%, and nearly 90% of customers indicated that the
taking more residents out on trips. The organization received transportation program made a difference in their ability to visit
101
Figure 6-4. Customer Satisfaction Tool

This survey tool was used to measure the satisfaction of families, friends, and outpatients using the transportation service.
1. Are the drivers courteous? Yes _____ No______
2. Do you feel safe with the driver? Yes _____ No______
3. Was the individual scheduling your transportation pleasant and

cooperative? Yes _____ No______
4. Were your transportation needs met? Yes _____ No______
5. Has transportation made a difference in your ability to visit (where

applicable)? Yes _____ No______
6. Did this program influence your decision to either admit yourself or your
family member/significant other (where applicable)? Yes _____ No______
7. Do you feel this program should continue? Yes _____ No______
Note: Please feel free to write comments on the back.
Source: Kings Harbor Multicare Center, Bronx, NY. Used with permission.
Figure 6-5. Resident Satisfaction Survey Tool

This survey tool was used to measure the satisfaction of residents using the transportation service.
1. Are the drivers courteous? Yes _____ No______
2. Do you feel safe with the driver? Yes _____ No______
3. Were your transportation needs met? Yes _____ No______
4. Has transportation made a difference in your ability to have visits, and

did the visits add to your feeling more like you were home (where
applicable)? Yes _____ No______
5. Do you feel this program should continue? Yes _____ No______
Note: Please feel free to write comments on the back.
Source: Kings Harbor Multicare Center, Bronx, NY. Used with permission.
102
the facility. In addition, more than 20% of responders indicated have been known to stop by a convenience store on the way
that the program influenced their decision to admit themselves back to the facility, so a resident can pick up a lottery ticket,”
or family members to the organization. “This program not only says Rizzo. “We also started putting car seats in the vans to
improved the satisfaction of our residents and their families, but accommodate visitors with small children, and we even accom-
it also helped with marketing. Prospective residents were modated one resident’s family’s seeing-eye dog.”
impressed by the fact that their transportation needs would be
met easily in the facility,” says Toni Mooney, vice president of Kings Harbor also saw an increase in staff satisfaction because
nursing for Kings Harbor. of the program. “We really involved staff members in design-
ing and implementing the program, and they found satisfac-
In addition to measuring resident and customer satisfaction, tion in seeing it work. They also feel that residents have
Kings Harbor also looked at the financial impact of the trans- responded better and participated more in their own rehabili-
portation program. Depreciation considerations allowed the tation since the program started,” says Rizzo.
organization to recoup the cost of the vans, and when compar-
ing the cost of hiring an outside firm to address the organiza- By listening to its residents, families, and staff members, Kings
tion’s transportation needs versus having in-house drivers Harbor was able to implement a program that not only
address those needs, the in-house approach was determined to increased resident and staff satisfaction but also helped
be more cost-effective. improve the organization’s ability to attract new residents. By
using data to measure the success of the program and the
Having full-time, consistent drivers also affected residents’ financial benefits associated with it, the organization has main-
quality of life. “The drivers get to know the residents and their tained the program and its level of resident satisfaction and
families and what their specific needs are. The drivers try to quality of care.
accommodate those needs when possible. For example, drivers
103
CASE STUDY 4:
HENRY FORD HEALTH SYSTEM
Depression affects about 10% of adults in the United States At first, the Blues Busters met as a separate leadership group to
every year. This illness can result in substantial medical care conduct improvement planning. The group provided regular-
expenditures, lost productivity, and absenteeism. If untreated ly structured updates to the executive team of the department
or treated poorly, depression can result in suicide. Each year, at of psychiatry and the monthly leadership group. However, as
least 1 in 10 people with depression takes his or her own life. goals and plans for the project were developed and integrated
into the overall strategic and operational plans of the depart-
The department of psychiatry at Henry Ford Health System ment, oversight responsibility for the initiative was transferred
(HFHS), a large integrated health system located in Detroit, to the executive team and monthly leadership group.
Michigan, recently conducted a PI initiative with the goal of
eliminating suicide among all patients within the department’s Setting Expectations
health maintenance organization (HMO) network. The In determining goals for the project and relevant performance
department of psychiatry has full responsibility for the mental measures, the Blues Busters team looked to the Institute of
health care of all 200,000 patients in this network. HFHS Medicine’s (IOM’s) Crossing the Quality Chasm1 report and set
sought to achieve its goal by targeting the initiative to reach goals for depression care that related to the report’s six quality
patients with depression and other mood disorders and by aims: safety, effectiveness, patient-centeredness, timeliness,
completely redesigning depression care delivery within its net- efficiency, and equity. (See Figure 6-6 on page 105.) HFHS
work. This initiative was implemented in collaboration with pursued a “zero defects” goal for the program, which in this
the Pursuing Perfection initiative sponsored by the Robert case meant completely eliminating patient suicide. Although
Wood Johnson Foundation and IHI. some team members initially balked at the idea of achieving
the stretch goal of eliminating patient suicides, the debate was
Prior to implementing the initiative, HFHS did provide good finally resolved when the question was asked, “If zero is not the
depression care. The organization followed patients closely right number of suicides, then what number is? 1? 4? 40?” The
during treatment and across the continuum of behavioral only answer that made sense was zero.
health care and used many evidence-based therapies. The
organization also used multidisciplinary teams and proactive Performance Improvement Activities
risk assessment to address performance. However, despite its Throughout the project, HFHS focused on improving its sys-
systematic approach, HFHS wanted to achieve further tem of behavioral health care, not simply on managing the par-
improvement in the area of patient suicide. ticular disease of depression. The following are some of the
many steps the organization took to reach its goal of perfect
Forming a Team depression care and improved system performance:
HFHS’s first step in addressing patient suicide was to form a PI • Partnered with patients to ensure that the voice of the cus-
team. The chair of the department of psychiatry created a steer- tomer was considered in care design and quality. To do this,
ing group to set the vision and goals for the project. This 15- HFHS established a consumer advisory panel and
member team included executive leadership, inpatient nursing, redesigned the treatment planning process with input from
physicians, therapists, and clinical managers. Both frontline staff the panel.
and leadership perspectives were represented. Early on, the team • Designed and implemented an evidence-based suicide pre-
adopted a name and logo, calling itself the “Blues Busters” and vention protocol that is used at both inpatient and outpa-
promising patients to “work with you to achieve the best possible tient sites. The clinical staff was trained on the protocol and
care, always respecting your individual wants and needs.” its use.
Developing the logo, name, and mission statement helped unite • Revised HFHS depression care guidelines to ensure system-
the team and gave it a defined purpose and identity within the atic and evidence-based use of psychotherapy, psychophar-
department and the larger health system. macology, and brain stimulation techniques
104
Figure 6-6. Project Key Goals and Indicators

This figure shows the goals, measures, and sources of data for the depression care initiative at Henry Ford Health System.
IOM Aim Goal Measure Data Source

Safety Eliminate inpatient falls Inpatient falls/1,000 days of care Incident reporting system
Eliminate inpatient medication errors Inpatient medication errors/1,000 days of care
Effectiveness Eliminate suicides Number of suicides/100,000 network members Incident reporting system
Patient- 100% of patients completely Overall patient satisfaction Press Ganey survey
centeredness satisfied with their care Assessment of Care survey
Timeliness 100% complete satisfaction Patient satisfaction with timeliness Assessment of Care survey
Efficiency 100% complete satisfaction Patient satisfaction with efficiency Assessment of Care survey
Equity 100% complete satisfaction Patient satisfaction with equity Assessment of Care survey
Source: Henry Ford Health System, Detroit, MI. Used with permission.
• Used a train-the-trainer approach to building departmen- in collecting and analyzing data. The organization collected
twide competency in cognitive behavior therapy suicide data for five years and also collected data related to the
• Designed and implemented clinical protocols to reduce the aforementioned IOM goals.
risk of falls and medication errors
• Provided patients with better access to treatment through HFHS used run charts to display the suicide data and also
drop-in group appointments, same-day access, and e-mail included the annual rate of suicide for the general population
visits. Each outpatient site provided weekly group appoint- in these run charts as a reference. Of note, the rate of suicide
ments that patients could drop in on. These were led by a in patients with active mood disorder is estimated at 80 to 90
psychiatrist and a social worker and provided temporary times the rate in the general population, and the suicide rate in
additional access and group support on short notice. patients with a history of suicide attempts is 100 times that of
• Improved information flow through the modification of an the general population.
electronic medical record, secure e-mail communication,
and an intranet site to disseminate depression guidelines to The PI team regularly reviewed results of the initiative with
all clinicians leaders and managers within the department of psychiatry. The
team used the department’s monthly newsletter, site visits to
HFHS tied the performance expectations of staff members in different HFHS locations, the organization’s semiannual edu-
with the goals of the depression care program. Progress toward cation and recognition day, and quarterly and annual reports
achieving predetermined goals and metrics that related to the to communicate with staff and leadership about the program
program was assessed informally through ongoing coaching and its results.
sessions and formally at semiannual and annual performance
evaluations. Thus, every member of the psychiatry department Results of the Program
had performance expectations that linked to achievement of For the first year of data collection, HFHS’s suicide rate was 85
the depression care project’s goals. This approach established per 100,000 patients. Although this rate was below expecta-
accountability for the staff and helped ensure buy-in. tions for a clinical population, the organization was still not
satisfied. In response to this information, HFHS designed,
Ensuring Effective Data Collection tested, and implemented multiple practice improvements,
Before starting to collect data, HFHS established a common such as refining the drop-in appointment, in order to contin-
definition of suicide, which every part of the organization used ue the positive results and further decrease the suicide rate.
105
Ultimately, the organization achieved a 71% reduction in its Busters team to accept “no suicides,” or zero defects, as the
patient population suicide rate. It has maintained this high goal. Striving for perfect depression care set the PI team and
level of improvement, such that the suicide rate of the organi- the entire department of psychiatry on a transformational
zation’s clinical population is several orders of magnitude journey. Perfect care required audacious goals, including trans-
below what would be expected in a clinical population. forming the behavior of everyone in the organization, from
leadership to frontline clinical and support staff members.
Lessons Learned Pursuing perfection became a core strategy for the organiza-
One of the most significant obstacles HFHS faced in imple- tion, fundamentally changing planning and operations and,
menting its program and realizing results was getting the Blues ultimately, performance.
106
CASE STUDY 5:
HOLY FAMILY MEMORIAL
Quality improvement is not something that happens ensure that the data being collected are appropriate and con-
overnight. It takes a concentrated, multifaceted approach that sistent and that there are no duplicative data collection efforts
involves effective and appropriate data collection, analysis, going on within the organization,” says Mary Schilder, Joint
interpretation, and response. Holy Family Memorial, a non- Commission resource coordinator at Holy Family Memorial.
profit integrated health system located in Manitowoc, To help with this, the quality department recently developed a
Wisconsin, has an award-winning approach to quality universal audit tool and guidelines to help direct a depart-
improvement. Sponsored by the Franciscan Sisters of Christian ment’s auditing process (see Figures 6-7 and 6-8 on pages 108
Charity, the organization has a 100-bed hospital, 16 physician and 109). The audit tool requires a department to state the
clinics, a home care program, and an outpatient behavioral nature of the requested audit and outline its expectations, its
health care program. Its quality department works with all results, and the actions that resulted from the audit. The
areas of the organization on data collection strategies, bench- guidelines offer strategies to use when designing measures,
marking, and action plan development. conducting audits, and interpreting results. “All audit work-
sheets are sent to the quality department for review. The qual-
A Quality Department Focused on Continuity ity department then determines whether another area of the
A key component of Holy Family’s quality approach is its qual- organization is already performing a similar audit that can be
ity department, which was recently reorganized and expanded used to address the requesting department’s need. In some
to promote and ensure continuity in quality improvement cases, a department can simply add a few questions to an exist-
efforts across the organization’s network of health care ing audit and obtain the necessary information without dupli-
providers. The department houses a variety of key people who cating work,” says Sue Shaw, R.N., director of continuity of
provide direction and support to the organization’s quality ini- care for Holy Family Memorial.
tiatives, including two quality specialists who are responsible
for the organization’s data collection and analysis efforts. Using Benchmarking to Drive Improvement
Although these two individuals do not collect all the data in The quality department at Holy Family Memorial relies on
the organization, they do serve as a resource for designing per- benchmarking to help improve quality and performance. “We
formance measures, ensuring consistency of data collection, tend to review available quality information from other hospi-
and appropriately interpreting data. tals that are similar to us in size, to see how we compare and
where we need to improve,” says Schilder. “For example, we
The quality specialists meet with various departments in the have done initiatives regarding unapproved abbreviations and
organization to determine what type of data need to be collect- immediately reportable critical test results in which we looked
ed on a particular issue and whether some data collection proj- at benchmarking numbers from other organizations, created
ects are no longer necessary. “In some departments, we have goals for improvement, and implemented programs to achieve
been collecting the same data for years, and we don’t really those goals.”
need to anymore. The quality specialists have been identifying
these outdated collection efforts and working to focus depart- Holy Family Memorial also uses internal benchmarking to
ments’ data collection projects on the most appropriate areas,” drive improvement. For example, the organization recently
says Marcia Donlon, R.N., administrative director of quality looked at the data it was collecting on pain management across
of care for Holy Family Memorial. The quality specialists also the organization. “I met with all the clinics and acute care
help with staff training on measure development, data collec- departments in our network to assess the level of compliance
tion, and data analysis and response. with our organization’s policies and protocols on pain mea-
surement, response, re-assessment, and documentation,” says
Ensuring Appropriate Data Collection Shaw. “Our goal for the network was to be at 90% compliance.
Holy Family collects data for PI through a variety of venues, If a particular department was not at 90% compliance, it was
including audits, observations, surveys, and current and retro- required to develop an action plan and work toward achieving
spective chart reviews. “The quality department works to that level of compliance.”
107
108
Figure 6-7. Audit Tool Worksheet
06-Chapter 6.qxp
The Holy Family Memorial quality department uses this audit tool to track all the different data collection activities taking place in the organization.
5/2/2008
9:44 AM
Page 108
Source: Holy Family Memorial, Manitowoc, Wisconsin. Used with permission. HFM, Holy Family Memorial.
Figure 6-8. Audit Guidelines

Holy Family Memorial offers these guidelines to departments starting an auditing project to ensure consistency in measurement
and accuracy in analysis.
AUDIT GUIDELINES
• Audit tool to be available on Intralink, HFM templates. Complete audit tool for each audit performed and send to
Quality Management.
• Audits should be performed and reported as indicated on Audit Tool Worksheet.
• Sample size is dependent upon census or volume of services rendered.
Recommendation:
❏ For a population size of less than 30 cases, sample 100%.
❏ For a population size of up to 100 cases, sample 30 cases.
❏ For a population size of 101 to 500 cases, sample 50 cases.
❏ For a population size of over 500 cases, sample 70 cases.
The population should be defined as any record, not just records of a specific case type.
• Analyze the data collected, identify problems, document, and take action to resolve the issue identified.
• Continue to monitor after action taken. Compile audit results and analyze data—taking further action if necessary.
• Measure of Success: A quantifiable measure that demonstrates whether an action was effective and sustained.
The actual level of sustained compliance, as averaged over a quarterly period, should be scored with the following
guidelines:
❏ 90%–100% = Satisfactory compliance.
❏ 80%–89% = Partial compliance—Two occurrences require corrective action.
❏ Less than 80% = Insufficient compliance—Immediate corrective action.
• Written policies and procedures aid in consistency of the audit process.
• Results of audits are to be sent to the Quality Management Department.
Demonstrating compliance:
• Documentation that shows audit is well-planned and corrective action taken when necessary.
• Results of audits and actions taken to improve must be in easy-to-read, graphical format.
Note: Expectation—Monthly audits to establish base. Once benchmark/goal achieved, audits every six months to
monitor compliance. If noncompliance occurs, quarterly audits resume until compliance. The exception of this
timetable is at the direction of a regulatory agency.
Quality specialists are available for education of the audit process.
Source: Holy Family Memorial, Manitowoc, Wisconsin. Used with permission. HFM, Holy Family Memorial.
109
Building Awareness of Safety Issues ment initiatives and supporting the quality improvement
The quality department works to build general awareness of department. “He recommended the expansion of the quality
patient safety and quality issues. “We try to keep our educa- department several months ago and has been a regular
tional efforts interesting and rewarding,” says Schilder. spokesperson to the board of directors and chief executive offi-
Recently, the organization created a “Room of Horrors.” This cer about why data collection and analysis are necessary and
was a designated room that was set aside for the education how a focus on quality improvement can make a difference in
project; it had 47 patient safety or quality violations within in patient safety, quality of care, and financial performance,” says
it, and many of these violations had to do with The Joint Schilder.
Commission’s National Patient Safety Goals. Some of the vio-
lations included having blood spilled on the floor and the side In addition, the CMO participates in the organization’s patient
rails of the patient bed in the down position. Outside the rounds routine. “It is really helpful for staff members to see
room, the organization had check sheets on which staff mem- senior leadership on the patient floor, talking with the staff and
bers could identify the things wrong in the demo room. Every patients. It not only makes the senior leader seem more
employee in the network was eligible to participate in the approachable and in touch with the organization’s needs, but
activity, and complete check sheets were sent to the quality it can secure a lot of buy-in for quality improvement initiatives
department for review. At the end of the education program, a and programs,” says Donlon.
$25 gift certificate was awarded to the individual who identi-
fied the most things wrong in the environment. “This program Holy Family Memorial looks at the quality process as a jour-
provided education and built awareness of patient safety issues, ney and is continually working on how to improve that process
and it was done in such a way that the staff enjoyed participat- throughout the organization. The quality department has
ing in it,” says Schilder. recently designed a quality improvement dashboard. “Our goal
is to make data and information transparent, so that anyone
Leadership Support Is Critical can access information about our quality efforts at any time,”
Another important contributor to Holy Family Memorial’s says Shaw. The dashboard helps easily communicate to staff,
success in quality improvement is the support of its senior leaders, patients, and so forth where the organization is in its
leadership. For example, the organization’s chief medical offi- journey toward quality care, including areas of success and
cer (CMO) has been instrumental in backing quality improve- those that need improvement.
Reference
1. Institute of Medicine: Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press, 2001.
110
07-Glossary.qxp 5/2/2008 9:45 AM Page 111
Glossary of Terms
accountability A situation in which all information is example, data indicating a unit’s compliance or
attributable to its source (person or device). noncompliance with hospital-established practice guidelines
would be attribute data.
accreditation Determination by The Joint Commission
that an eligible health care organization complies with audit (of data quality) Regular focused inspections by the
applicable Joint Commission standards. measurement system of a sample of participating
organization-level records and processes to ensure the
accuracy (of data) The extent to which data are free of
accuracy and completeness of measurement data on
identifiable errors.
performance.
adverse drug event (ADE) A patient injury resulting
auditability Performance measure data obtained from
from a medication, either because of a pharmacological
enrolled health care organizations are traceable at the
reaction to a normal dose or because of a preventable
individual case level so that performance measurement
adverse reaction to a drug resulting from an error.
systems can adequately assess the quality of data.
aggregate data Measurement data collected and reported
automated database A computerized system that
by organizations as a sum or total over a given time period
processes data electronically and provides for data storage
(for example, monthly, quarterly) or for certain groupings
and access through electronic interface.
(for example, health care organization level).
baseline An observation or a value that represents the
analysis The process of interpreting data and transforming
current background level of a measurable quantity, as
them into information.
in a baseline rate for performance of ultrasonography
attribute data Data that arise from the classification of during pregnancy (expressed as the number of procedures
items into categories, from counts of the number of items per patient per pregnancy). The baseline rate is used for
in a given category or the proportion in a given category, comparison with values representing responses to
and from counts of the number of occurrences per unit. For experimental intervention or an environmental stimulus,
113
usually implying that the baseline and response values refer to common cause variation Variation in a process that is due
the same individual or system. to the process itself and is produced by interactions of
variables of that process. Common cause variation is inherent
benchmark A point of reference or standard by which
in all processes; it is not a disturbance in the process. It can
something can be measured, compared, or judged, as in
be altered only by changing the process.
benchmarks of performance.
comparative (across organizations) measure-related
benchmarking Continuous comparison of an organization’s
feedback Information comparing the performance of
process, product, or service to that of the toughest
organizations in a measurement system to each other
competitor, to those considered industry leaders, or to similar
(interorganizational) on measure-related data, (for example,
activities in the organization in order to find and implement
indicator rates, ratios, indices, percentages) according to
ways to improve it. See also competitive benchmarking,
predefined, standardized parameters.
external benchmarking, functional benchmarking, generic
benchmarking, internal benchmarking. comparison group The group of health care organizations
to which an individual health care organization is compared.
bias Any systematic error in the design, conduct, or analysis
of a study that results in a mistaken estimate of effect. competitive benchmarking A type of internal
benchmarking in which an organization’s processes, products,
calculation algorithm An ordered sequence of data element
or services are compared to those of the best competition in
retrieval and aggregation through which numerator and
the industry.
denominator events or continuous variable values are
identified by a measure. configuration (of a performance measurement system)
The operational characteristics of a performance
central tendency A property of the distribution of a
measurement system.
variable, usually measured by statistics such as the mean,
median, and mode. continuous variable data Data for which many possible
results are possible (for example, length of stay, minutes to
clinical measures Measures designed to evaluate the
treatment).
processes or outcomes of care associated with the delivery of
clinical services; allow for intra- and interorganizational control chart A graphic display of data in the order in
comparisons to be used to continuously improve patient which they occur, with statistically determined upper and
health outcomes; may focus on the appropriateness of clinical lower limits of expected common cause variation. A control
decision making and implementation of these decisions; must chart is used to identify special causes of variation, to
be condition specific, be procedure specific, or address monitor a process for maintenance, and to determine
important functions of patient care (for example, medication whether process changes have had the desired effect.
use, infection control, patient assessment).
control limit In statistics, an expected limit of common
clinical practice guidelines According to the Institute of cause variation, sometimes referred to as either an upper or a
Medicine, systematically developed statements to assist lower limit. Variation beyond a control limit is evidence that
practitioners and patients in choosing appropriate health care special causes are affecting a process. Control limits are
for specific clinical conditions. Guidelines reflect current calculated from process data and are not to be confused with
scientific knowledge of practices and expert clinical judgment engineering specifications or tolerance limits. Control limits
on the best ways to prevent, diagnose, treat, or manage are typically plotted on a control chart.
diseases and disorders.
correlation A relationship between statistical variables that
clinical/service groups (CSGs) Groups of patients in tend to vary, be associated, or occur together in a way that is
distinct, clinical populations for which data are collected. not expected on the basis of chance alone.
clinical survey (data source) Survey data obtained from criteria Expected levels of achievement or specifications against
clinicians who provide care. which performance or quality may be compared. (For example,
114
Glossary of Terms
criteria for appropriate initial care of a patient with a headache denominator data elements Those data elements required
may be a measurement of body temperature and blood pressure to construct the denominator.
and performance of a neurological examination.)
development (of a performance measurement system)
data Uninterpreted measurements, facts, or clinical The process of growing by degrees into a more advanced or
observations collected during an assessment activity. Data mature system; the planned promotion of understanding,
may be quantitative (numeric) or qualitative (reflecting the cultivation, participation, and support for a measurement
“lived experience” of the population of interest). system.
database An organized, comprehensive collection of data dimensions of performance Nine definable, measurable,
elements (variables) and their values. and improvable attributes of performance related to “doing
the right things right” (appropriateness, availability, and
data collection The process of capturing raw or primary
efficacy) and “doing things well” (timeliness, effectiveness,
data from a single source or a number of sources.
continuity, safety, efficiency, and respect and caring).
data editing The process of correcting erroneous or
distribution In statistics, the complete summary of the
incomplete existing data, exclusive of data entry input edits.
frequencies of the values or categories of measurement made
data element A discrete piece of data, such as patient birth on a group of persons or other entities. The distribution
date or principal diagnosis. See also denominator data determines either how many or what proportion of a group
elements, numerator data elements, and risk adjustment data was found to have each value (or each range of values) out of
elements. all possible values that the quantitative measure can have. A
bell-shaped curve, or normal distribution, is an example of a
data maintenance The efforts required to keep database
distribution in which the greatest number of observations
files and supporting documentation accurate.
falls in the center, with fewer and fewer observations falling
data point The representation of a value for a set of evenly on either side of the average.
observations or measurements at a specific time interval (for
effectiveness The degree to which care, treatment, and
example, surgical site infection rate for the month of
services are provided in the correct manner, given the current
December 2008).
state of knowledge, to achieve the desired or projected
data security The protection of data from intentional or outcome(s) for an individual.
unintentional destruction, modification, or disclosure.
efficacy The degree to which care is provided in the correct
data sources The primary source document(s) used for data manner, given the current state of knowledge, to achieve the
collection (for example, billing or administrative data, desired or projected outcome(s).
encounter form, enrollment forms, medical record). The
efficiency The relationship between the outcomes (results of
developer may choose to standardize the data source(s) used
care) and the resources used to deliver care.
for data collection across organizations, or may allow data to
be collected from several sources at the participating external benchmarking The comparison of similar
organizations' discretion. processes with other organizations.
defined measure A structured measure with defined external data source A repository for data that exists
populations that measure specific events or values; such outside of the measurement system's control.
measures may have numerators and denominators, take the
failure mode and effects analysis (FMEA) An assessment
form of a continuous variable, or result from survey
that proactively examines a process in detail, including
questions.
sequencing of events; assesses actual and potential risk,
denominator The lower part of a fraction used to calculate a failure, or points of vulnerability; and, through a logical
rate, proportion, or ratio. Also the population for a rate- process, prioritizes areas for improvement based on the actual
based measure. or potential patient care impact (criticality).
115
flowchart A pictorial summary that shows with symbols and measurement system for use in the organization's ongoing
words the steps, sequence, and relationship of the various efforts to improve patient care and organization performance.
operations involved in the performance of a function or a Feedback can be reflective of information within individual
process. organizations (intraorganizational) and/or across
organizations (interorganizational).
functional benchmarking A type of external benchmarking
in which an organization compares performance of a work median The value in a group or set of ranked observations
function with that of a leader (usually) not in the same that divides the data into two equal-sized parts (that is, the
industry. middle number of a data set). The median is the most valid
measure of central tendency when a distribution is skewed.
general data elements The group of data elements used to
link health care organization level to comparison group data. MedPAR (Medicare Provider Analysis and Review) data
Data collected by the Centers for Medicare & Medicaid
generic benchmarking A type of external benchmarking in
Services (CMS) from hospitals in order for those hospitals to
which an organization compares best practices of a process or
receive reimbursement for performed services and procedures.
function across industries, wherever they occur.
The data elements are defined by Medicare billing
good practices Clinical, scientific, or professional practices requirements.
that are recognized by a majority of professionals in a
medical record (data source) Data obtained from the
particular field. These practices are typically evidence based
records or documentation maintained on a patient in any
and consensus driven.
health care setting (for example, hospital, home care, long
health status measures Measures that address the term care, practitioner office). Includes automated and paper
functional well-being of specific populations, both in general medical record systems.
and in relation to specific conditions, demonstrating change
medication error Any preventable event that may cause
over time (for example, physical functioning, bodily pain,
inappropriate medication use or jeopardize patient safety.
social functioning, mental health).
medication management The process an organization uses
histogram A graphic display, using a bar graph, of the
to provide medication therapy to individuals served by the
frequency distribution of a variable.
organization. The steps in the medication management
indicator A measure used to determine, over time, an process include selection, procurement, storage, prescribing
organization’s performance of functions, processes, and or ordering, transcribing, preparing, dispensing,
outcomes. administration, self-administration, and monitoring.
internal benchmarking The comparison of similar processes minimum data set An agreed-on and accepted set of terms
within an organization. and definitions constituting a core of data; a collection of
related data items.
mean A measure of central tendency for a continuous
variable measure. The mean is the sum of the values divided mode In statistics, a measure of central tendency of a
by the number of observations. collection of data that consists of the measurement of the
data set that occurs most often.
measure A valid and reliable indicator that can be used to
monitor and evaluate the quality of important governance, monitoring Performance measurement systems have an
management, clinical, and support functions that affect ongoing data quality monitoring process in place so that data
patient outcomes. quality is continuously monitored based on pre-specified data
quality standards. It may be in the form of a periodic internal
measurement The systematic process of data collection,
data quality study or on-site data auditing.
repeated over time or at a single point in time.
normal distribution See distribution.
measure-related feedback Measure-related information on
performance that is available, on a timely basis, to numerator The upper portion of a fraction used to calculate
organizations actively participating in the performance a rate, proportion, or ratio.
116
Glossary of Terms
numerator data elements Those data elements necessary or performance measurement See measurement.
required to construct the numerator.
performance measurement system An entity consisting
ongoing data quality review process A data quality of an automated database(s) that facilitates performance
review process that is currently in operation and that is improvement in health care organizations through the
intended to continue for as long as data are accepted into the collection and dissemination of process and/or outcome
database. measures of performance. Measurement systems must be able
to generate internal comparisons of organization performance
outcome The result of the performance (or
over time and external comparisons of performance among
nonperformance) of a function(s) or process(es).
participating organizations at comparable times.
outcome measure A measure that indicates the result of the
performance measure–related feedback See measure-
performance (or nonperformance) of a function(s) or
related feedback.
process(es).
practice guidelines Tools that describe processes found by
outlier An observation that differs so widely from the rest of
clinical trials or by consensus opinion of experts to be the
a data set as to suggest that a gross error may have been
most effective in evaluating and/or treating a patient who has
committed or that the variation is due to a special, rather
a specific symptom, condition, or diagnosis, or in describing
than a common, cause.
a specific procedure. Synonyms include practice parameter,
Pareto chart A form of vertical bar graph that displays protocol, preferred practice pattern, and guideline.
information in such a way that priorities for process
priority focus areas (PFAs) Processes, systems, or
improvement can be established. It shows the relative
structures in a health care organization that can significantly
importance of all the data and is used to direct efforts to the
impact the quality and safety of care.
largest improvement opportunity by highlighting the vital
few in contrast to the many others. Priority Focus Process (PFP) The process for
standardizing the priorities for sampling during an
patient survey (data source) Survey data are exclusively
organization’s on-site survey based on information collected
obtained from patients and/or their family
about the organization prior to survey. The process also helps
members/significant others.
to focus the survey on areas that are critical to that
percentile A value on a scale of 100 that indicates the organization’s patient safety and quality of care processes.
percentage of a distribution that is equal to or below it. Examples of such information may include, but are not
limited to, data from the organization’s electronic application,
perception of care/service measures Satisfaction
compliance and sentinel event information, and data
measures that focus on the delivery of clinical care from the
collected from external sources.
patient's/family's/caregiver's perspective. These include, but
are not limited to, the following aspects of patient care: process An interrelated series of events, activities, actions,
patient education, medication use, pain management, mechanisms, or steps that transform inputs into outputs.
communication regarding plans and outcomes of care,
process measure A measure that focuses on a process that
prevention and illness, improvement in health status, and so
leads to a certain outcome, meaning that a scientific basis
forth. A measure may address one or more aspects of care.
exists for believing that the process, when executed well, will
performance improvement The continuous study and increase the probability of achieving a desired outcome.
adaptation of a health care organization’s functions and
protected health information Health information that
processes to increase the probability of achieving desired
contains information such that an individual person can be
outcomes and to better meet the needs of individuals and
identified as the subject of that information.
other users of services.
provider data (data source) Data obtained from other
performance measure A quantitative tool (for example,
provider-generated records that are not necessarily contained
rate, ratio, index, percentage) that provides an indication of
in the medical record (for example, pharmacy patient
an organization's performance in relation to a specified
medication profiles, nursing care plans).
process or outcome. See also measure.
117
proximate cause An act or omission that naturally and risk adjustment model The statistical algorithm that
directly produces a consequence. It is the superficial or specifies the numerical values and the sequence of
obvious cause of an occurrence or event. calculations used to risk adjust (such as, reduce or remove the
influence of confounding factors) performance measures.
random In statistics, relating to the same or equal chances or
probability of occurrence for each member of a group. For run chart A display of data in which data points are plotted
example, in clinical research, the probability of assignment of as they occur over time (for example, observed weights over
a given subject to a specified treatment group is fixed and time) to detect trends or other patterns and variation
constant (typically 0.50), but the subject’s actual assignment occurring over time. Run charts, as opposed to tabular
cannot be known until it occurs. frequency displays, are capable of time-order analytic studies.
range A measure of the spread of a data set; the difference sampling The process of selecting a group of units, portions
between the smallest and largest observation. of material, or observations from a larger collection of units,
quantity of material, or observations that serves to provide
rate Derived by dividing the numerator (for example, cases
information that may be used as a basis for making a decision
that meet the criterion for good or poor care) by the
concerning the larger quantity.
denominator (for example, all cases to which the criterion
applies) within a given time frame. In other words, the sampling method If data are sampled, the sample size is
numerator is a subset of the denominator. determined based on power analysis and the individual cases
in the population have equal opportunities for being sampled
ratio A relationship between two counted sets of data, which
so that the sample data adequately represent the population.
may have a value of zero or greater. In a ratio, the numerator
is not necessarily a subset of the denominator (for example, satisfaction measures Measures that address the extent to
pints of blood transfused to number of patients discharged). which the patients/enrollees, practitioners, and/or purchasers
perceive their needs to be met (for example, provider and
reliability The ability of the indicator to accurately and
service delivery, administrative and financial aspects, overall
consistently identify the events it was designed to indentify
satisfaction). See also perception of care measures.
across multiple health care settings.
sentinel event An unexpected occurrence involving death or
retrievability The capability of efficiently finding relevant
serious physical or psychological injury or the risk thereof.
information.
Serious injury specifically includes loss of limb or function.
risk-adjusted rate A rate that takes into account differences The phrase or the risk thereof includes any process variation
in case mix to allow for more valid comparisons between for which a recurrence would carry a significant chance of a
groups. serious adverse outcome.
risk adjustment A statistical process for reducing, removing, special cause A factor that intermittently and
or clarifying the influences of confounding factors that differ unpredictably induces variation over and above that inherent
among comparison groups; mostly used for outcome in the system. It often appears as an extreme point, such as a
measures to adjust for confounding patient factors in order to point beyond the upper and lower control limits on a control
have fairer comparisons. chart, or as some specific and identifiable pattern in data.
risk adjustment data elements Those data elements used special cause variation The variation in performance and
to risk adjust a performance measure (such as, reduce, data that results from special causes. Special cause variation is
remove, or clarify the influences of confounding patient intermittent, unpredictable, and unstable. It is not inherently
factors that differ among comparison groups). Such data present in a system; rather, it arises from causes that are not
elements may be used exclusively for risk adjustment (such part of the system as designed. It tends to cluster by person,
as, not required to construct the numerator or denominator) place, and time and should be eliminated by an organization
or may be required for numerator or denominator if it results in undesirable outcomes.
construction as well as for risk adjustment.
118
Glossary of Terms
staffing effectiveness The number, competence, and skill structure measure A measure that assesses whether
mix of staff as related to the provision of needed services. organizational resources and arrangements are in place to
deliver health care, such as the number, type, and
standard deviation A measure of variability that indicates
distribution of medical personnel, equipment, and facilities.
the spread of a set of observations around the mean.
validity The extent to which a measure accurately reflects the
standard of quality A generally accepted, objective standard
concept or construct that it is intended to measure.
of measurement, such as a rule or guideline, supported
through findings from expert consensus, based on specific validity and reliability Performance measurement systems
research and/or documentation in scientific literature, against ensure that individual data accurately measure what they
which an individual's or organization's level of performance purport to measure in a consistent manner regardless of data
may be compared. abstractors, data collection methods, or data sources within
and across health care organizations.
statistical process control The application of statistical
techniques, such as control and comparison charts, to analyze variance A measure of the differences in a set of
a process or its output so as to take appropriate actions to observations.
achieve and maintain a state of statistical control and to
variation The differences in results obtained in measuring
improve the capability of the process.
the same phenomenon more than once. The sources of
stratification A form of risk adjustment that involves variation in a process over time can be grouped into two
classifying data into subgroups based on one or more major classes: common causes and special causes.
characteristics, variables, or other categories.
119
08-Index.qxp 5/2/2008 9:47 AM Page 121
08-Index.qxp 5/2/2008 9:47 AM Page 122
08-Index.qxp 5/2/2008 9:47 AM Page 123
Index
A Anticoagulant therapy, reduction of adverse events in, 94–97
Accountability, 6, 113 Application for accreditation, electronic (e-App), 86, 88
Accreditation Participation Requirements, 85 Attribute data, 35–36, 113
Accreditation process, 6, 7, 85–90, 113 Audit of data quality, 13, 22, 113
electronic application for accreditation in, 86, 88 in Holy Family Memorial health system, 107, 108, 109
evidence of standards compliance in, 89 Automated database, 113
on-site survey in, 85, 87–89
Periodic Performance Review in, 85–86 B
Priority Focus Process in, 86–87 Balanced scorecard approach, 15, 49–50
role of data management in, 6, 85 Baldrige model of performance improvement, 14–16
tracer methodology in, 87–89 Bar charts, 38, 40, 49
Accuracy of data, 113 Baseline, 113–114
Admission records as data source, 20 Bell-shaped curve in normal distribution, 37, 115
Adverse events, 113 Benchmarking, 114
importance of consistent terminology in, 26 competitive, 114
near miss in, 20–21 external, 115
Novant Health case study on, 94–97 functional, 116
reporting systems in, 20–21 generic, 116
Affinity diagrams, 59 in Holy Family Memorial health system, 107
Agency for Healthcare Research and Quality, 4, 5, 25, 67 internal, 107, 116
Aggregate data, 113 secondary data in, 20
Algorithms, 114 Bias, 114
in process improvement and redesign, 68 in surveys, 24
American Society for Quality, 52 Billing records as data source, 20
Analysis of data, 12–13, 35–53, 113. See also Data analysis Black belts in Six Sigma approach, 52, 82
123
08-Index.qxp 5/2/2008 9:47 AM Page 124
Blues Busters team in Henry Ford Health System, 104–106 Comparison analysis, 44–48
Brainstorming in performance improvement process, 58, 66 levels of, 45
Brungs, Suzanne, 98 methods of, 47–48
risk adjustment in, 45, 47
C Comparison group, 114
Calculation algorithm, 114 Competency assessment of staff, 80
Care maps, 68, 69–77 Competitive benchmarking, 114
Case studies, 93–110 Complaint data, 21
on adverse event reduction in Novant Health, 94–96 Compliance with The Joint Commission standards, 6, 85–90
on Baldrige model in Saint Luke's Hospital, 14–16 Evidence of Standards Compliance (ESC) report on, 89
on control chart use, 45 Measure of Success (MOS) in, 86, 89
on infection control in Greater Cincinnati Patient Safety ICU on-site survey for assessment of, 87–89
Collaborative, 97–100 Periodic Performance Review (PPR) of, 85–86
on quality improvement in Holy Family Memorial health Concept mapping, 59
system, 107–110 Concurrent data collection, 26
on satisfaction with transportation services at Kings Harbor Confidence intervals, 48, 49
Multicare Center, 101–103 Configuration of performance measurement system, 114
on suicide prevention in Henry Ford Health System, 104–106 Consumer Assessment of Healthcare Providers and Systems (CAHPS)
on testing and implementation of performance improvement Survey, 25
plan, 79 Continuous variables, 36, 114
Categorical variables, 35 central tendency measures for, 37
kappa statistic for, 30 intraclass correlation statistics for, 30
Catheter-related infection control in Greater Cincinnati Patient Safety standard deviation of, 38, 39
ICU Collaborative, 97–100 X-bar and S charts on, 44, 45, 46
Causal loop diagrams, 59 XmR charts on, 44, 45, 46
Cause-and-effect diagrams, 59, 60, 66 Control charts, 42–44, 49, 114
Centers for Medicare & Medicaid Services, 4 case example on, 45
Home Health Quality Initiative, 7 control limits in, 44, 114
Nursing Home Quality Initiative, 6 types of, 44, 45, 46
pay-for-performance initiatives, 4, 5 variation in, 37, 42, 44
Central tendency, 37, 114 Control limits, 44, 114
Certification on health care quality and improvement issues, 52 Correlation, 115
Certified Professional in Healthcare Quality, 52 Cost-benefit analysis, 65
Charter or charge statement of performance improvement team, 64 Cost-effectiveness analysis, 65
Checklists for data collection, 23 Cost-utility analysis, 65
Clinical measures, 10, 114 Criteria, 114
Clinical pathways, 21, 68 for exclusion or inclusion, 27
care maps of, 68, 69–77 Crossing the Quality Chasm: A New Health System for the 21st Century
Clinical practice guidelines, 78, 114 report (Institute of Medicine), 4, 104
Clinical/service groups, 114 Culture, organizational
individual tracers in, 88, 89 in Baldrige model, 15
in Priority Focus Process, 86–87, 88 learning in, 9
Clinical surveillance as data source, 20 quality in, 9
Clinical survey, 114 Customer satisfaction survey on Kings Harbor Multicare Center
Cluster sampling, 28 transportation services, 101, 102
Collection of data, 10–12, 19–31, 115. See also Data collection
Common cause variation, 36, 114 D
in control charts, 42 Dashboards, 13, 49, 50–51
in run charts, 42, 43 in Holy Family Memorial health system, 110
Communication resources allocated to performance improvement in Novant Health, 96
teams, 66 Data, definition of, 115
Comparative measure-related feedback, 114 Data access of performance improvement team, 65
124
08-Index.qxp 5/2/2008 9:47 AM Page 125
Index
Data analysis, 12–13, 35–53, 113 overview of, 3–16

comparison analysis in, 44–48 in performance improvement process, 6–8, 13, 57–82
of external data, 12 sharing of data in, 13, 23
of internal data, 12–13 in Greater Cincinnati Patient Safety ICU Collaborative,
presentation and organization of information in, 13, 48–51 98–99
in Greater Cincinnati Patient Safety ICU Collaborative, in Novant Health, 96
98–99 planning for, 23
in Novant Health, 96 system tracers on, 89
statistical techniques in, 35–38 Data point, 115
tools for, 38–44 Data presentation, 13, 48–51
training programs on, 12–13, 51–52 in Greater Cincinnati Patient Safety ICU Collaborative, 98–99
Data collection, 10–12, 19–31, 115 in Novant Health, 96
annual review of, 12 Data security, 22–23, 115
concurrent, 26 Data sources, 10–11, 12–13, 115
in Greater Cincinnati Patient Safety ICU Collaborative, 98, 99, clinical survey as, 114
100 electronic, 21
in Henry Ford Health System, 105 examples of, 20–21
in Holy Family Memorial health system, 107 external, 12, 31, 115
in Kings Harbor Multicare Center, 101, 102 internal, 12–13
performance measures in, 11–12, 22, 24–29 medical records as, 20, 116
pilot tests in, 23 patient survey as, 117
planning for, 11, 21–23 provider data as, 117
primary and secondary data in, 19–20 Database, 115
problem identification prior to, 11 automated, 113
quality of data in, 22, 29–31 on sentinel events, 67
retrospective method, 26 Define-measure-analyze-improve-control (DMAIC) method, 81
sampling methods in, 12, 27–29 Defined measures, 12, 115
security of data in, 22–23 Deming, W. Edwards, 81
software for, 29 Denominators, 115
sources of data in, 10–11, 12, 20–21 in Greater Cincinnati Patient Safety ICU Collaborative case
tools used in, 23–24 study, 99
training on, 22 in performance measure construction, 11, 27
types of data in, 19–20 in rate calculation, 37, 118
validity and reliability in, 12, 22, 29–31 in ratio calculation, 26, 118
interrater or interobserver, 29, 30 Depression care in Henry Ford Health System, 104–106
Data editing, 115 Design-measure-analyze-design-verify (DMADV) method, 81
Data elements, 115, 116 Development of performance measurement system, 11–12, 115
denominator, 115 Dimensions of performance, 115
numerator, 117 Distribution, 115
risk adjustment, 118 normal bell-shaped curve in, 37, 115
well-defined, 12 outliers in, 47–48, 117
Data maintenance, 115 percentiles in, 47, 117
Data management standard deviation in, 37–38, 119
in accreditation process, 6, 85 Donlon, Marcia, 107
in analysis of data, 12–13, 35–53
in Baldrige model of performance improvement, 14 E
in collection of data, 10–12, 19–31 Education and training. See Training programs
dashboards or report cards in, 13, 49, 50–51, 96, 110 Effectiveness, 115
goals of, 8–9 compared to costs in performance improvement, 65
leadership support and involvement in, 9, 10 of staffing, 119
national focus on, 4–6 Efficacy, 115
organizational resources required for, 10 Efficiency, 115
125
08-Index.qxp 5/2/2008 9:47 AM Page 126
Electronic application for accreditation (e-App), 86, 88 on quality of health care, 4

Electronic data sources, 21 reliability of, 118
Elements of performance (EPs), 86, 89 Individual tracers, 87–89
Errors in health care, 4 Infection control
in medications, 94–97, 113, 116 in Greater Cincinnati Patient Safety ICU Collaborative, 97–100
Evidence of Standards Compliance (ESC) report, 89 system tracers on, 89
Exclusion criteria, 27 Information access for performance improvement team, 65
External benchmarking, 115 Institute for Healthcare Improvement, 5–6, 67
External data sources, 12, 31, 115 Breakthrough Series training, 97
5 Million Lives Campaign, 6
F 100,000 Lives Campaign, 5
Failure mode and effects analysis (FMEA), 59, 66–67, 115 Institute of Medicine
Federal government role in quality of health care, 4–5 Crossing the Quality Chasm: A New Health System for the 21st
Feedback, measure-related, 116 Century report, 4, 104
comparative, 114 To Err is Human: Building a Safer Health System report, 4
Financial issues Internal benchmarking, 107, 116
data collection on, 11 Internal data sources, 12–13
in Kings Harbor Multicare Center transportation services, 103 International Normalized Ratio in anticoagulant therapy, 94–96
performance measures on, 24, 65 Intraclass correlation statistics, 30
in resource allocation to performance improvement teams, 63, Ishikawa diagrams, 59, 60
65
Fishbone diagrams, 59, 60 J
Flowcharts, 66, 116 Joint Commission, The, 4
Force-field analysis, 78 accreditation process, 6, 7, 85–90
Forms for data collection, 23 data collection requirements, 6, 7
Functional benchmarking, 116 extranet site of, 87, 88
in quality movement, 6
G Sentinel Event Database, 67
Good practices, 116 standards of, 6, 7, 85–90
Greater Cincinnati Health Council, 97, 99 Strategic Surveillance System (S3), 87, 88
Greater Cincinnati Patient Safety ICU Collaborative Journals and newsletters on quality improvement, 52
case study on, 97–100 Judgment sampling, 28–29
hospitals participating in, 97
Green belts in Six Sigma approach, 52, 82 K
Kaplan, Robert, 15, 50
H Kappa statistic, 30
Hawthorne effect, 29 Kings Harbor Multicare Center case study, 101–103
Health status measures, 116
Henry Ford Health System case study, 104–106 L
Histograms, 38, 42, 116 Leadership roles, 9, 10, 57–58
continuous variables in, 36 in Baldrige model, 14, 15
Holy Family Memorial health system case study, 107–110 in Holy Family Memorial health system, 110
Home Health Quality Initiative of CMS, 7 in Six Sigma approach, 81–82
Hospice care, performance and quality measures in, 24 in team approach, 61, 62, 63
Hospital Consumer Assessment of Healthcare Providers and Systems The Leapfrog Group, 4
(HCAHPS), 25 Learning, in performance improvement process, 9
Hospital Quality Alliance, 5, 25 Lederer, James, 94, 96
Length of stay data, 13
I standard deviation in, 40
Inclusion criteria, 27 Line graphs, 38, 41, 49
Indicators, 116 Logbooks as data collection tool, 23
126
08-Index.qxp 5/2/2008 9:47 AM Page 127
Index
M Norton, David, 15, 50

Malcolm Baldrige National Quality Award program, 14–16 Nosocomial infection control in Greater Cincinnati Patient Safety
Manager of Quality/Organizational Excellence, certified, 52 ICU Collaborative, 97–100
Mean Novant Health case study, 94–96
as measure of central tendency, 37, 116 Numerators, 116, 117
standard error of, 48 in Greater Cincinnati Patient Safety ICU Collaborative case
Measure of Success (MOS), 86, 89 study, 99
Measure-related feedback, 116 in performance measure construction, 11, 27
comparative, 114 in rate calculation, 37, 118
Measurement, definition of, 116 in ratio calculation, 26, 118
Measurement error, 29 Nursing Home Quality Initiative of CMS, 6
in surveys, 24
Measures, 116 O
defined measures, 12, 115 Observations
on health status, 116 as data source, 20
identification and development of, 11–12, 22, 24–29 in evaluation of data quality, 29
on outcome, 12, 24, 26, 117 Organizational resources
on perception of care/service, 11, 117 access of performance improvement team to, 61, 63–66
on performance. See Performance measures required for data management, 10
on process, 24, 26, 117 structure measures on, 26, 119
on satisfaction, 11, 24, 118. See also Satisfaction measures Outcome measures, 24, 26, 117
sensitivity and specificity of, 29 in external data, 12
on structure, 26, 119 in hospice care, 24
validity and reliability of, 29, 119 Outliers, 117
Median, as measure of central tendency, 37, 116 Z-score for, 47–48
Medical records, 20, 116
electronic, 21 P
Medicare, pay-for-performance initiatives in, 4, 5 P-charts, 44, 45, 46
Medicare Health Care Quality Demonstration Program, 5 Pareto charts, 58–59, 66, 117
Medicare Provider Analysis and Review (MedPAR) data, 86, 88, 116 Patient survey as data source, 117
Medication errors, 116 Pay-for-performance initiatives of CMS, 4, 5
adverse drug events in, 94–97, 113 Percentiles, 47, 117
Medication management, 116 Perception of care/service measures, 11, 25, 117
Novant Health case study on, 94–97 Performance improvement, 117
system tracers on, 89 in accreditation process, 86
MedPAR (Medicare Provider Analysis and Review) data, 86, 88, 116 Baldrige model of, 14–16
Meetings of performance improvement teams, 65–66 data management in, 6–8, 13, 57–82
Minimum data set, 116 failure mode and effects analysis in, 66–67
Mock tracers, 59, 89 in Greater Cincinnati Patient Safety ICU Collaborative, 97–100
Mode, as measure of central tendency, 37, 116 in Henry Ford Health System, 104–106
Monitoring, 116 in Holy Family Memorial health system, 107–110
of data quality, 31, 117 in Kings Harbor Multicare Center, 101–103
of performance improvements, 80 leadership roles in, 9, 10. See also Leadership roles
Mooney, Toni, 103 monitoring of, 80
Mortality reports, 21 in Novant Health, 94–96
Multivoting process, 60 plan-do-study-act approach to, 13, 44, 78–80, 94
preparation for, 61–66
N priorities in, 9, 57–61, 62, 63
National Association for Healthcare Quality, 52 redesign of processes for, 67–78
National Patient Safety Goals, 85, 86, 110 root cause analysis in, 67
National Quality Forum, 11, 25 Six Sigma in, 81–82
Newsletters and journals on quality improvement, 52
127
08-Index.qxp 5/2/2008 9:47 AM Page 128
staff resistance to initiatives in, 68, 78 ongoing measurement and monitoring of, 80
team approach to. See Team approach to performance plan-do-act-act approach in, 44, 78–80
improvement redesign of process in, 67–78
testing and implementation of plan for, 78–80 root cause analysis in, 67
case study on, 79 Six Sigma in, 81–82
in Novant Health, 94 statistical control in, 44, 119
Performance measurement, 117 testing and implementation of plans for, 78–80
configuration of system for, 114 in Novant Health, 94
development of system for, 11–12, 115 Process measures, 24, 26, 117
national focus on, 4–6 Proportions, p-charts on, 44, 45, 46
ongoing, 80 Protected health information, 117
requirements on, 7 Provider data, 117
validity and reliability in, 12, 119 Proximate cause, 118
Performance measures, 117
in Baldrige model, 15 Q
identification and development of, 11–12, 22, 24–29 Quality auditor, certified, 52
trends over time in, 13 Quality engineer, certified, 52
Performance Risk Assessment in Strategic Surveillance System, 88 Quality improvement associate, certified, 52
Periodic Performance Review, 85–86 Quality of data, 22, 29–31
Physician Group Practice Demonstration Program, 5 audit of, 13, 22, 113
Physician roles, 10 in Holy Family Memorial health system, 107, 108, 109
Pie charts, 38, 41, 49 ongoing review of, 31, 117
Pilot tests standard of, 119
in data collection, 23 Quality of health care, 3
on performance improvement plan, 78 continuous improvements efforts in Holy Family Memorial
Plan-do-study-act (PDSA) process, 13, 44, 78–80 health system, 107–110
in Novant Health, 94 federal government role in, 4–5
Planning for data collection, 11, 21–23 indicators of, 4
Practice guidelines, 117 national focus on, 4–6
Premier Hospital Quality Incentive Demonstration Program, 5 The Joint Commission role in, 6
Presentation of data, 13, 48–51
in Greater Cincinnati Patient Safety ICU Collaborative, 98–99 R
in Novant Health, 96 Random sample, 27, 28, 29, 118
Primary data, 19 simple, 28, 29
Priorities for performance improvement, 9, 57–61, 62, 63 systematic, 28, 29
Prioritization matrix, 60–61, 62 Range of data set, 118
weighted, 61, 63 compared to standard deviation, 37
Priority focus areas (PFAs), 86–87, 88, 117 Rank order, 47
in tracer methodology, 89 Rate, 118
Priority Focus Process (PFP), 86–87, 117 central tendency calculation for, 37
quarterly reports on, 87, 88 risk adjusted, 118
and tracer methodology, 87, 89 Ratio, 26, 118
Probability sampling methods, 28 u-charts on, 44, 45, 46
Problem identification Reliability, 118, 119
failure mode and effects analysis in, 66–67 in data analysis, 13
prior to data collection, 11 in data collection, 12, 22, 29–31
root cause analysis in, 67 interrater or interobserver, 29, 30
Process, definition of, 117 Report cards on multiple measures, 13, 51
Process improvement Reporting systems
in Baldrige model, 14 on adverse events, 20–21
failure mode and effects analysis in, 66–67
128
08-Index.qxp 5/2/2008 9:47 AM Page 129
Index
on mortality and complications, 21 Sharing of data, 13

Resident satisfaction survey on Kings Harbor Multicare Center in Greater Cincinnati Patient Safety ICU Collaborative, 98–99
transportation services, 101, 102 in Novant Health, 96
Resistance to performance improvement initiatives, 68, 78 planning for, 23
Response rate in surveys, 24 Shaw, Sue, 107
and nonresponse bias, 24 Shewhart charts, 44
Retrievability of information, 118 Shift reports as data source, 20
Retrospective data collection, 26 Significance, statistical, 48
Risk adjustment, 118 Simulation exercises on performance improvement plan, 78
in comparison analysis, 45, 47 Six Sigma, 13, 81–82
in external data, 12 black belts in, 52, 82
stratification in, 119 green belts in, 52, 82
Risk assessment in Strategic Surveillance System, 88 Software for data collection, 29
Rizzo, Hilary, 101, 103 Special cause variation, 36–37, 118
Root cause analysis (RCA), 67 in control charts, 37, 42, 44
cause-and-effect diagrams in, 59 in run charts, 42
Run charts, 38, 42, 43, 118 tests for identification of, 36–37, 44
interpretation of, 42, 43 Specificity of measures, 29
variations in, 42, 43 Staffing effectiveness, 119
Standard deviation, 37–38, 119
S calculation of, 38, 39–40
S charts, 44, 45, 46 for continuous data, 38, 39
Saint Luke's Hospital, Baldrige model of performance improvement Z-score in, 47–48
in, 14–16 Standard error of the mean, 48
Sampling methods, 12, 27–29, 118 Standard of quality, 119
cluster, 28 Standards of The Joint Commission, 6, 7, 85–90
errors in, 24, 27 Statistical techniques, 35–48
implementation of, 29 in process control, 44, 119
judgment, 28–29 in reliability evaluation, 30
random, 27, 28, 29, 118 sampling methods in, 27–29
size of sample in, 28 significance in, 48
stratified, 28 training programs on, 51–52
in surveys, 24 Strategic planning in Baldrige model, 14
Satisfaction measures, 11, 24, 118 Strategic Surveillance System (S3), 87, 88
in Consumer Assessment of Healthcare Providers and Systems Stratification
survey, 25 in risk adjustment, 119
data collection on, 11, 21 in sampling method, 28
on Kings Harbor transportation services, 101–103 Stress tests on performance improvement plan, 78
Scatter diagrams, 38 Structure measures, 26, 119
Schilder, Mary, 107, 110 Suicide prevention in Henry Ford Health System, 104–106
Secondary data, 19–20 Surgical site infection control in Greater Cincinnati Patient Safety
sensitivity of, 20 ICU Collaborative, 97–99
Security policies, 22–23, 115 Surveillance data, 20
Selection bias, 24 Survey, on-site, in accreditation process, 6, 85, 87–89
Sensitivity tracer methodology in, 87–89
of measures, 29 Surveys as data source, 23–24
of secondary data, 20 bias in, 24
Sentinel events, 118 clinical survey in, 114
near miss in, 20–21 Consumer Assessment of Healthcare Providers and Systems
reporting systems in, 20–21 (CAHPS) survey in, 25
The Joint Commission database on, 67 errors in, 24
Severity adjustment in external data, 12 patient survey in, 117
129
08-Index.qxp 5/2/2008 9:47 AM Page 130
response rate in, 24 U

sampling methods in, 24 U-charts, 44, 45, 46
on satisfaction with Kings Harbor Multicare Center Utility, compared to costs in performance improvement, 65
transportation services, 101, 102
System tracers, individual-based, 87, 89 V
Validity, 119
T in data analysis, 13
T-statistics, 48 in data collection, 12, 22, 29–31
Tables for data analysis, 38, 48 Variability, standard deviation in, 37–38, 119. See also Standard
Team approach to performance improvement, 61–80 deviation
access to organizational resources in, 61, 63–66 Variation, 37–38, 119
in Baldrige model, 15 common cause, 36, 42, 43, 114
characteristics of successful team in, 64 in control charts, 37, 42, 44
charter or charge statement in, 64 in run charts, 42, 43
composition of team in, 61–62 special cause, 36–37, 42, 44, 118
failure mode and effects analysis in, 66–67 Virtual performance improvement teams, 65–66
goal identification in, 62–63
in Greater Cincinnati Patient Safety ICU Collaborative, 97 W
in Henry Ford Health System, 104, 105 Warfarin therapy, reduction of adverse events in, 94–97
in Kings Harbor Multicare Center, 101 Web site resources
leadership role in, 61, 62, 63 on Consumer Assessment of Healthcare Providers and Systems
in Novant Health, 94 survey, 25
redesign of process in, 67–78 of Greater Cincinnati Health Council, 99
root cause analysis in, 67 of The Joint Commission, 86, 87, 88
testing and implementation of plans in, 78–80 on quality improvement journals and newsletters, 52
virtual teams in, 65–66 Weighted prioritization matrixes, 61, 63
The Joint Commission. See Joint Commission, The
To Err is Human: Building a Safer Health System report (Institute of X
Medicine), 4 X-bar charts, 44, 45, 46
Tracer methodology, 20, 87–89 XmR charts, 44, 45, 46
individual tracers in, 87–89
mock tracers in, 59, 89 Z
system tracers in, individual-based, 87, 89 Z-scores, 47–48
Training programs
on data analysis, 12–13, 51–52
on data collection, 22
in Holy Family Memorial health system, 110
for performance improvement, 80
Transportation services of Kings Harbor Multicare Center, 101–103
130

Managing Performance Measurement Data in Health Care

Uploaded by

Copyright:

Available Formats

Managing Performance Measurement Data in Health Care

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Managing Performance Measurement Data in Health Care

Uploaded by

Copyright:

Available Formats

PMHC51 COVER FINAL.

qxp 5/2/2008 9:29 AM Page 1

Managing Performance Measurement Data

Managing Performance Measurement Data in Health Care, Second Edition

This updated book also contains the following special features:

About Joint Commission Resources

Order Code: PMHC51

Joint Commission Resources (JCR),

1515 West 22nd Street, Suite 1300W

Managing Performance Measurement Data

Executive Editor: Jaclyn Graham

Joint Commission Resources Mission

© 2008 by the Joint Commission on Accreditation of Healthcare Organizations

Printed in the U.S.A. 5 4 3 2 1

Chapter 1: Data Management: An Overview ................................................................................................3

Chapter 2: Data Collection ................................................................................................................................19

Managing Performance Measurement Data in Health Care, Second Edition

Chapter 3: Analyzing Data ......................................................................................................................................35

Chapter 5: A Data-Driven Accreditation Process ..........................................................................................85

Chapter 6: Case Studies—A Spotlight on Success ........................................................................................93

I n many health care organizations, the idea of data manage-

Managing Performance Measurement Data in Health Care, Second Edition

Setting-specific examples in this book use the appropriate terminology.

• Kings Harbor Multicare Center, Bronx, New York

Thanks are also extended to Kathleen B. Vega for her tireless

Data Management: An Overview

A n organization cannot improve its performance if it does

Managing Performance Measurement Data in Health Care, Second Edition

Chapter 1: Data Management: An Overview

Managing Performance Measurement Data in Health Care, Second Edition

Chapter 1: Data Management: An Overview

Managing Performance Measurement Data in Health Care, Second Edition

Chapter 1: Data Management: An Overview

Managing Performance Measurement Data in Health Care, Second Edition

Organization Resources The Steps to Effective Data

Chapter 1: Data Management: An Overview

Managing Performance Measurement Data in Health Care, Second Edition

Chapter 1: Data Management: An Overview

Managing Performance Measurement Data in Health Care, Second Edition

Understanding the Criteria

Chapter 1: Data Management: An Overview

Sidebar 1-6, continued

Creating a Culture That Supports the Management Philosophy

Aligning Organization Performance Measures with Strategic Goals

Managing Performance Measurement Data in Health Care, Second Edition

Sidebar 1-6, continued

Source: St. Luke’s Hospital. Used with permission.

Summary 6. Agency for Healthcare Research and Quality: AHRQ Profile.

T he process of data collection is a critical part of the data

Managing Performance Measurement Data in Health Care, Second Edition

Chapter 2: Data Collection

Managing Performance Measurement Data in Health Care, Second Edition

Chapter 2: Data Collection

Managing Performance Measurement Data in Health Care, Second Edition

Chapter 2: Data Collection

Managing Performance Measurement Data in Health Care, Second Edition

The selection of performance measures should be done sys-

Chapter 2: Data Collection

Managing Performance Measurement Data in Health Care, Second Edition