THE COLLEGE OF LE COLLÈGE DES

FAMILY PHYSICIANS MÉDECINS DE FAMILLE

OF CANADA DU CANADA

BEST ADVICE

Advanced and Meaningful

Use of EMRs
MODULE 1
Using EMRs for Quality
Improvement and Research

NOVEMBER 2018
Increasingly, family physicians have the capacity to improve the quality of care they deliver using information
contained in electronic medical records (EMRs). The measurement of key indicators and outcomes provides crucial
insights into a clinical practice and can provide a solid evidence base for quality improvements (QI) to better meet
the needs of patients and communities. This module describes QI, how physicians can use EMRs to improve care
through QI methodologies, and outlines other advanced uses of EMR data for QI and research.

WHAT IS QUALITY IMPROVEMENT?

At its core, QI encompasses several formal approaches to understanding performance and systematically improving
the processes of patient care delivery.1 In the health care system, there are always opportunities to optimize,
streamline, develop, and test processes to improve efficiency, patient safety, and clinical outcomes. QI is about
continually making incremental changes to improve systems and thereby outcomes.

There is a variety of approaches that can be used to collect and analyze data, test changes, and implement
improvements. The following section describes two approaches to QI in more detail: the model for improvement, and
audit and feedback.

THE MODEL FOR IMPROVEMENT

The model for improvement consists of two basic components: answering three fundamental questions (before the
QI initiative can take place), and conducting the rapid cycle improvement process, comprised of a series of Plan-
Do-Study-Act (PDSA) cycles (Figure 1).2 The PDSA cycle involves testing a change by developing a plan to test the
change (plan), carrying out the test (do), observing and learning from the consequences (study), and determining
what modifications should be made to the test (act).3

Figure 1: Model for Improvement

Source: Langley GJ, Moen RD, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to
Enhancing Organizational Performance, 2nd Edition. San Francisco, CA: Jossey-Bass, a Wiley Imprint; 2009. Reproduced with permission.

2  Best Advice – Advanced and Meaningful Use of EMRs Module 1  NOVEMBER 2018
 PDSA resources
The Institute for Healthcare
Improvement (IHI) provides
More specifically, conducting PDSA cycles involves the following steps: 4
practical tools and instructions for
1. Forming a QI team conducting PDSA cycles in your
practice or organization, including:
2. Setting aims • How to Improve: Information
3. Establishing measures about each of the seven steps
of a PDSA cycle, with examples
4. Selecting changes • PDSA Cycles (part 1, part 2):
Videos explaining how to use
5. Testing changes
the PDSA cycle
6. Implementing changes • Plan-Do-Study-Act (PDSA)
Worksheet
7. Spreading changes Note: You may need to create a
free account on the IHI website to
access some of the resources.

Source: Created by Håkan Forss. @hakanforss; hakanforss.wordpress.com. Reproduced with permission.

In a busy practice it is important to be realistic about what family physicians can reasonably accomplish with QI.
For a typical family physician, the PDSA approach offers a highly adoptable model to consider using for small
improvement cycles.

EXAMPLE OF QI PROCESS USING A PDSA CYCLE

Following are highlights of a case study in the Health Quality Ontario publication Quality Improvement Guide.5

The new director of a rehabilitation hospital noted that the hospital’s falls rate was much higher than that of similar hospitals.
Looking at the data, staff saw that the unit with the highest number of falls looked after relatively mobile residents
who had mild to moderate dementia and were receiving stroke rehab.

Over several meetings, the team:

• Offered suggested changes to address the issue
• Set the project aim (a 40 per cent improvement of desired metric)
• Implemented a falls risk assessment using an existing tool
• Used assessment results to apply prevention protocols (balance training, medication review)

NOVEMBER 2018  Advanced and Meaningful Use of EMRs Module 1 – Best Advice  3 
 Clinical audit tools
The Irish College of General
• Included the assessments in training for all new staff Practitioners clinical audit website
• Measured the fall rate and noted trends that led to the 40 per cent has practical tools and instructions
for conducting clinical audits in a
reduction
practice or an organization.
• Shared the assessment methodology, and their success, with other
units

A full description of this case study is available in the guide.5

AUDIT AND FEEDBACK APPROACH

Audit and feedback is defined as “any summary of clinical performance of health care over a specified period of time
aimed at providing information to health professionals to allow them to assess and adjust their performance.”6 It is
an overarching term that describes some of the measures used to improve professional practice. Health professionals
can receive feedback about their performance based on data derived from their routine practice (and through EMRs).
Physicians should not be concerned about the term “audit” as this process represents a reflective exercise that
physicians can initiate in their own practices. Audit stages include:7
1. Choosing a topic
2. Defining aims and objectives
3. Choosing guidelines, stating criteria, and setting a standard
4. Collecting data (from the EMR)
5. Analyzing and interpreting data
6. Deciding what changes need to be made and implementing them

GENERAL QI RESOURCES

Table 1 provides additional practical resources that can help you get started with QI in your practice. Table 2
contains additional literature about QI.

4  Best Advice – Advanced and Meaningful Use of EMRs Module 1  NOVEMBER 2018
Table 1: General QI resources

Resource Description
Quality Improvement in Healthcare – A short and engaging video about the basics of QI and its role in health care.
Dr. Mike Evans

Quality Improvement Guide – A comprehensive guide about QI that includes the following sections:
Health Quality Ontario • Background of QI
• Case examples
• Implementing PDSA cycles
• Sample worksheets

Practice Improvement Initiative Pii uses quality improvement, practice-level data, and research to improve
(Pii) – College of Family Physicians everyday practice, the patient experience, efficiences, and the work experience
of Canada of health care providers (the Quadruple Aim). It enables family physicians,
residents, and the teams they work in to undertake QI within their practices and
to use practice-level data. To support practices, Pii has developed a Practice
Improvement Essentials (PIE) Part 1 workshop, which introduces the concept of
QI, explores practical models and tools. There will also be a follow-up workshop
called PIE Part 2, which is a practical, case-based application of QI, data, and
research readiness.

Data Boot Camp: How to Get A webinar series that delves deeper into QI. The free one-hour webinars include
Your Data Into Shape – Canadian information about:
Foundation for Healthcare • How to assess the quality of data from various sources, including electronic data
Improvement (e.g., EMRs) and manually-collected data (e.g., audits, time-and-motion studies)
• Tools for improving data quality
• How to use data effectively—measuring the right things the right way,
performing necessary data preparation, and processing and disseminating data

CPD eCoach – University of An online self-directed assessment tool for QI that involves the following four steps
British Columbia Continuing and includes specific information about EMR data extraction for common EMRs:
Professional Development in 1. Define your topic.
collaboration with the BC College
of Family Physicians 2. Conduct self-directed assessment.
3. Create a plan for improvement.
4. Evaluate the implementation.

NOVEMBER 2018  Advanced and Meaningful Use of EMRs Module 1 – Best Advice  5 
Table 2: General QI literature

Resource Description
Impact of a primary healthcare An evaluation of an Ontario primary care QI program about diabetes
quality improvement program on
diabetes in Canada: evaluation
of the Quality Improvement and
Innovation Partnership (QIIP)8
Effective strategies for scaling Review of strategies to implement and scale up evidence-based practices in
up evidence-based practices in primary care
primary care: a systematic review9
Systematic review of the application Review of the plan-do-study-act cycle’s impact in QI
of the plan-do-study-act method to
improve quality in healthcare10

Achieving change in primary Literature review of implementation strategies for changing professional practices
care—effectiveness of strategies
for improving implementation of
complex interventions: systematic
review of reviews11
Reporting and design elements of Describes key elements of the audit and feedback approach
audit and feedback interventions:
a secondary review12

USING EMRS FOR QI

The previous section provided a general background about QI and some common QI methodologies used in health
care. The following section describes how to use EMRs for QI.

Improvements in EMRs have resulted in an expanded ability to use more comprehensive data to support QI
initiatives. Because EMRs hold a wide variety of data, they can be quite useful for collecting information, measuring
change, and collecting feedback.13

DATA QUALITY: CAN YOU MEASURE WHAT YOU WANT TO IMPROVE?

To perform QI using an EMR, you must have a good understanding of the data your system collects. Based on that
assessment, you will have a better idea of the types of indicators you can use for QI. We cannot improve what we
cannot measure, so consistently recording patient health data is crucial. Some examples of metrics you may be able
to audit, depending on your EMR’s capacity, include:14,15
• Appointment processes: Time from request to appointment, cancelled appointments/no shows,
third next available appointment
• Patient encounters: Chief complaint
• Patient communication: Patient reminders, patient education
• Medication management: Medication list/reconciliation, drug interaction/allergy alterations

6  Best Advice – Advanced and Meaningful Use of EMRs Module 1  NOVEMBER 2018
DATA QUALITY FOR QI
To perform a small-scale test of a PDSA cycle, consider the following questions:
• What are we trying to improve?
• How can it be measured?
• Is the EMR capable of capturing the data we want to measure?

Imagine that you want to improve specific aspects of care for diabetic patients in your practice. Using the questions
above, you might identify the following:
• What are we trying to improve: Hypertension in type 2 diabetic patients
• How can it be measured: By consistently taking and recording patients’ blood pressure (BP) and by
identifying patients above the target BP level
• Is the EMR capable of capturing the data we want to measure: Yes

The first step might be to create a list of all diabetic patients in your practice, ensuring the list is accurate and desired
data is captured consistently. If you are confident about the quality of your EMR data, you can start with a PDSA
exercise using the questions suggested above. If not, consider refining your research question until you are able
to collect the necessary EMR data. Improving (or simply assessing) the data quality in your EMR is often the most
important place to start when pursuing QI in your practice.16

Once you identify the possibilities for QI projects based on available data from your EMR you can undertake a PDSA
or an audit and feedback project. EMRs are the data source for these endeavours and can be used to pull patient lists,
run queries, and track progress over time. A recent Cochrane review on this topic concluded that audit and feedback
interventions, “generally lead to small but potentially important improvements in professional practice.”

NOVEMBER 2018  Advanced and Meaningful Use of EMRs Module 1 – Best Advice  7 
QI is enabled by high-quality data and physicians should ensure that their EMR data entry policies result in
standardized and consistent records. Free text entry of information that introduces inconsistencies as the same
condition can be described in many different terms (e.g., blood pressure can be entered as B/P, BP, Sys/Dias, or HT).
These inconsistencies serve as obstacles for automated analysis of data that is often used in QI.

Here are a few tips to ensure higher quality EMR data:

• Use drop-down lists rather than free text where possible
• Use built-in EMR features, where possible, that allow filling in medical history fields with industry standard
terms such as ICD9, ICD 10, or SNOMED-CT. A clinician may add any key elements that are not part of
standardized nomenclature manually.
• Use the keyword search feature of your EMR to help test the quality of the data. Having to use different
search words to identify all the patients in your practice with a certain diagnosis or issue is an indicator that
there are quality gaps.

COMMUNICATING RESULTS OF QI PROJECTS AND MANAGING CHANGE

Communicating the results of QI projects is a critical aspect of all QI initiatives. Teams and organizations may resist change,
so it is important to engage them in the process from the start, and think about approaches for communicating results of QI
projects to ensure changes are implemented and sustained. The resources in Table 3 provide some useful information.

ADVANCED USES OF EMR DATA FOR PRACTICE IMPROVEMENT

PRACTICE BASED RESEARCH NETWORKS

Over the past decade practice based research networks (PBRNs) have been collecting primary care data for disease
surveillance, research, and supporting QI efforts at a macro scale. Canada has one of the world’s largest networks,
the Canadian Primary Care Sentinel Surveillance Network (CPCSSN).20

The CPCSSN is Canada’s multi-disease surveillance system based on primary care EMR data collected via regional
networks in seven provinces and one territory. In total, the system includes data representing over 1.5 million patients
associated with over 1,000 primary care providers.21 The power of PBRNs, such as the CPCSSN, has been recognized
by Canada Health Infoway and other international organizations.22 The value for physicians to participate in PBRNs
goes beyond the personal gratification of contributing to research. The direct benefit can be seen in both the opportunity
to engage in research and to use CPCSSN’s practice feedback reports. These reports allow physicians to compare their
practice to others based on the data provided. To learn more about joining one of CPCSSN’s regional networks and
receiving regular reports on your practice, or about accessing data for research and QI projects, visit CPCSSN’s website.

8  Best Advice – Advanced and Meaningful Use of EMRs Module 1  NOVEMBER 2018
Table 3: Resources about communicating results of QI initiatives

Resource Description
Practice Feedback Interventions: Learn how to best communicate the results and outcomes of QI initiatives.
15 Suggestions for Optimizing The article includes suggestions to improve the effectiveness of feedback and
Effectiveness17 examples of associated implementation strategies, such as:
• Recommend actions that are consistent with established goals
• Recommend actions that are under the recipient's control
• Recommend specific actions
• Provide multiple instances of feedback
• Address credibility of the information provided
• Prevent defensive reactions to feedback

Advancing the literature on Learn how to effectively provide feedback (based on QI projects) to clinicians.
designing audit and feedback For example:
interventions: identifying theory-
• Use noun descriptors rather than verbs in messaging; for example, "do not be
informed hypotheses18
an over prescriber” versus “please prescribe less”
• Involve recipients’ feedback in the design of the QI initiative from the
beginning

Ten challenges in improving Learn about 10 common challenges in QI and opportunities to overcome them;
quality in healthcare: lessons for example:
from the Health Foundation's
• Convincing team members that there actually is a problem
programme evaluations and
relevant literature19 • Dealing with tribalism and lack of staff engagement
• Ensuring the changes are sustainable

PATIENT’S MEDICAL HOME AND PRAGMATIC CLINICAL TRIALS

The concept of data collection, research, and ongoing QI is a crucial piece of the Patient’s Medical Home (PMH)—the
CFPC’s vision of family practice in Canada. In family medicine, the laboratory that will drive research that is relevant to
practice is the PMH, powered by EMR data. The CFPC has strongly endorsed the importance of convenient access to
EMR data for QI and research purpose, encouraging vendors to make this information available.23

Pragmatic, or practical, clinical trials (PCTs) are defined as “trials for which the hypothesis and study design are
based on the information needed to make a decision.”24 They are different than explanatory clinical trials, which try
to better understand how and why an intervention works. PCTs, on the other hand, address practical questions about
risks, benefits, and costs of an intervention as they would occur in a routine clinical practice. Table 4 lists some key
PCT resources. PCTs can be conducted within PMH practices, using EMR data.

There are four key features of PCTs:24

• Compares clinically relevant alternative interventions, rather than a treatment with no treatment/placebo
• Enrols a diverse study population
• Recruits from a variety of practice settings
• Measures a broad range of relevant health outcomes

NOVEMBER 2018  Advanced and Meaningful Use of EMRs Module 1 – Best Advice  9 
Table 4: Resources for pragmatic clinical trials

Resource Description
Practical clinical trials: increasing the value of A seminal paper that contains information about the key
clinical research for decision making in clinical characteristics of PCTs, and strategies for improving the supply of
and health policy24 PCTs in the health system.

Practical clinical trials for translating research Contains specific recommendations and examples for designing
to practice: design and measurement and conducting PCTs, selecting appropriate measures for the trials,
recommendations25 and reporting the results.

The randomized registry trial—The next An article that discusses randomized registry trials that use large
disruptive technology in clinical research?26 scale data sources (i.e., combined EMR data) to conduct randomized
experiments at a much lower cost than traditional trials.

Atrial fibrillation anticoagulation care in a large An example of a PCT conducted in a family medical practice in Canada.
urban family medicine practice27

PRACTICE LINKED REPORTS

Several provinces are home to health quality councils (e.g., Ontario, Saskatchewan, Alberta, and British Columbia)
that send reports linking practice data to administrative health data. These reports allow family physicians to see how
practice populations interact with the health system beyond EMR-based data. To sign up for a report tailored to your
practice, visit the relevant health quality organization’s website (e.g., HQO My Practice Primary Care in Ontario).

SOCIAL DETERMINANTS OF HEALTH

Another example of how EMR data linkages could revolutionize care in Canada is managing social determinants of health
in clinical practice. Statistics Canada’s postal code data can be converted into accurate measures of income quintile and
social deprivation for clinic populations. The potential to use this data more meaningfully with EMR-based clinical data
offers many opportunities to reveal issues of poverty and marginalization in practice populations. To learn more about
addressing social accountability in your practice refer to the Best Advice guide Social Determinants of Health.

CONCLUSION

Among their many benefits, EMRs can support family physicians in conducting QI and research to improve patient care.
Family physicians shouldn’t feel overwhelmed by QI. Even in a busy practice, physicians can start small by using EMRs
to answer simple questions about the practice and patients, and then work to make incremental changes to improve
patient care using PDSA cycles or audit and feedback approaches. On a broader scale, EMR data can be used for large-
scale research through PBRNs and conducting PCTs. Harness the data collected through EMRs and take advantage of
QI and research opportunities to improve your practice, patient outcomes, and the health system more broadly.

10  Best Advice – Advanced and Meaningful Use of EMRs Module 1  NOVEMBER 2018
 References
1. American Academy of Family Physicians. Basics of Quality Improvement website. www.aafp.org/practice-management/improvement/basics.html. Accessed 2018 June 20.

2. Health Quality Ontario. Quality Improvement Guide. Toronto, ON: Health Quality Ontario; 2012. Available from: www.hqontario.ca/portals/0/Documents/qi/
qi-quality-improve-guide-2012-en.pdf. Accessed 2018 June 20.

3. Institute for Health Improvement. Plan-Do-Study-Act (PDSA) Worksheet website. www.ihi.org/resources/Pages/Tools/PlanDoStudyActWorksheet.aspx. Accessed
2018 June 20.

4. Institute for Health Improvement. How to Improve. Available from: www.ihi.org/resources/Pages/HowtoImprove/default.aspx. Accessed 2018 June 20.

5. Health Quality Ontario. “QI Case #1.” In Quality Improvement Guide, 6.Toronto, ON: Health Quality Ontario; 2012. Available from: www.hqontario.ca/portals/0/
Documents/qi/qi-quality-improve-guide-2012-en.pdf. Accessed 2018 June 20.

6. Flottorp SA, Jamtvedt G, Gibis B, McKee M. Using audit and feedback to health professionals to improve the quality and safety of health care. Copenhagen,
Denmark: World Health Organization; 2010. Available from: www.euro.who.int/__data/assets/pdf_file/0003/124419/e94296.pdf. Accessed 2018 June 20.

7. Irish College of General Practitioners. Clinical Audit website. www.icgp.ie/go/pcs/scheme_framework/clinical_audit. Accessed 2018 June 20.

8. Reichert SM, Harris SB, Tompkins JW, Belle-Brown J, Fourner M, Green M, et al. Impact of a primary healthcare quality improvement program on diabetes in
Canada: evaluation of the Quality Improvement and Innovation Partnership (QIIP). BMJ Open Diabetes Res Care. 2017;5(1):e000392.

9. Charif AB, Zomahoun HTV, LeBlanc A, Langlois L, Wolfenden L, Yoong SL, et al. Effective strategies for scaling up evidence-based practices in primary care: a
systematic review. Implement Sci. 2017;12(1):139.

10. Taylor MJ, McNicholas C, Nicolay C, Darzi A, Bell D, Reed JE. Systematic review of the application of the plan-do-study-act method to improve quality in
healthcare. BMJ Qual Saf. 2014;23(4):290-298.

11. Lau R, Stevenson F, Ong BN, Dziedzic K, Treweek S, Eldridge S, et al. Achieving change in primary care—effectiveness of strategies for improving implementation
of complex interventions: systematic review of reviews. BMJ Open. 2015;5(12):e009993.

12. Colquhoun H, Michie S, Sales A, Ivers N, Grimshaw JM, Carroll K, et al. Reporting and design elements of audit and feedback interventions: a secondary review.
BMJ Qual Saf. 2017; 26(1): 56-60.

13. Greiver M, Drummond N, Birtwhistle R, Queenan J, Lambert-Lanning A, Jackson D. Using EMRs to fuel quality improvement. Can Fam Phys. 2015; 61(1): 92.

14. MedPro Group. Guideline: Using an Electronic Health Record System as a Quality Improvement Tool. MedPro Group: 2016. Available from: www.medpro.com/
documents/10502/2837997/Guideline_Using+an+EHR+as+a+Quality+Improvement+Tool.pdf. Accessed 2018 June 20.

15. Institute for Healthcare Improvement. Third Next Available Appointment website. www.ihi.org/resources/Pages/Measures/ThirdNextAvailableAppointment.aspx.
Accessed 2018 September 7.

16. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes.
Cochrane Database of Systematic Reviews. 2012; Issue 6. Art. No.: CD000259.

17. Brehaut JC, Colquhoun HL, Eva KW, Carroll K, Sales A, Michie S, et al. Practice Feedback Interventions: 15 Suggestions for Optimizing Effectiveness. Ann Intern
Med. 2016;164(6): 435–441.

18. Colquhoun HL, Carroll K, Eva KW, Grimshaw JM, Ivers N, Michie S, et al. Advancing the literature on designing audit and feedback interventions: identifying
theory-informed hypotheses. Implement Sci. 2017; 12(117).

19. Dixon-Woods M, McNicol S, Martin G. Ten challenges in improving quality in healthcare: lessons from the Health Foundation's programme evaluations and
relevant literature. BMJ Qual Saf. 2012;21:876-884.

20. Gentil ML, Cuggia M, Figuet L, Hagenbourger C, Le Berre T, Banatre A, et al. Factors influencing the development of primary care data collection projects from
electronic health records: a systematic review of the literature. BMC Med Inform Decis Mak. 2017; 17(1):139.

21. CPCSSN. Canadian Primary Care Sentinel Surveillance Network website. Available from: cpcssn.ca. Accessed 2018 June 20.

22. Canada Health Infoway. Emerging Technology Series: Clinical Analytics in Primary Care (White Paper). Toronto, ON: Canada Health Infoway; 2016. Section D. Available from:
www.infoway-inforoute.ca/en/component/edocman/resources/reports/2882-clinical-analytics-in-primary-care-white-paper-full-report?Itemid=101. Accessed 2018 June 20.

23. College of Family Physicians of Canada. New CFPC Position Statement Supports access to EMR data for quality improvement and research. Mississauga, ON:
College of Family Physicians of Canada; 2017. Available from: www.cfpc.ca/uploadedFiles/Publications/News_Releases/News_Items/CFPC-News-Release-CFPC-
Statement-EMR-Data-Access-EN.pdf. Accessed 2018 June 20

24. Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003;290(12):1624-1632.


25. Glasgow RE, Magid DJ, Beck A, Ritzwoller D, Estabrooks PA. Practical clinical trials for translating research to practice: design and measurement recommendations.
Med Care. 2005;43(6):551-557.

26. Lauer MS, D'Agostino RB. The randomized registry trial--the next disruptive technology in clinical research? N Engl J Med. 2013;369(17):1579-1581.

27. Valentinis A, Ivers N, Bhatia S, Meshkat N, Leblanc K, Ha A, et al. Atrial fibrillation anticoagulation care in a large urban family medicine practice. Can Fam
Physician. 2014; 60(3): e173-e179.

NOVEMBER 2018  Advanced and Meaningful Use of EMRs Module 1 – Best Advice  11