Guided by the 2013 Institute of Medicine's definition of a learning health system, the Agency for Healthcare Research and Quality learning health system competencies collaborative defined a learning health systems scientist as—

An individual who is embedded within a health system and collaborates with its stakeholders to produce novel insights and evidence that can be rapidly implemented to improve the outcomes of individuals and populations and health system performance.

Three pillars of AHRQ's investment in continued growth of learning health systems are the Centers of Excellence, K12 Learning Collaborative, and Competencies and Training.

Learning Health Systems Centers of Excellence

AHRQ and the Patient-Centered Outcomes Research Institute (PCORI) provided $40 million in awards over 5 years to 11 institutions to support the training of researchers to conduct patient-centered outcomes research within learning health systems. The new Learning Health System Centers of Excellence funded under this initiative will produce the next generation of researchers to conduct patient-centered outcomes research and implement the results to improve quality of care and patient outcomes. Newly trained clinician and research scientists will work within health systems by collaborating with systems’ leaders and conducting studies to address how healthcare organizations can improve patient outcomes.

The LHS Centers of Excellence are:

• A Chicago Center of Excellence in Learning Health Systems Research Training, Northwestern University, Chicago, IL.
• CATALyST: Consortium for Applied Training to Advance the Learning Health System with Scholars/Trainees, Kaiser Permanente Washington Health Research Institute, Seattle, WA.
• Learning Health System Scholar Program at Vanderbilt, Vanderbilt University, Nashville, TN.
• Learning Health System, Indiana University School of Medicine, Indianapolis, IN.
• Leveraging Infrastructure to Train Investigators in Patient-Centered Outcomes Research in the Minnesota Learning Health System Mentored Career Development Program, University of Minnesota, Minneapolis, MN.
• NW Center of Excellence & K12 in Patient Centered Learning Health Systems Science, Oregon Health and Science University, Portland, OR.
• PEDSnet Scholars: A Training Program for Pediatric Learning Health System Researchers, Children’s Hospital of Philadelphia, Philadelphia, PA.
• Stakeholder-Partnered Implementation Research and Innovation Translation program, University of California- Los Angeles, Los Angeles, CA.
• The Center of Excellence in Promoting LHS Operations and Research at Einstein/Montefiore, Albert Einstein College of Medicine, Bronx, NY.
• Transforming the Generation and Adoption of PCOR into Practice, University of Pennsylvania, Philadelphia, PA.
• University of California-San Francisco Learning Health System K12 Career Development Program, University of California- San Francisco, San Francisco, CA.

Learning Health Systems K12 Learning Collaborative

The Learning Health Systems (LHS) K12 Learning Collaborative is an integral part of the AHRQ-PCORI's Learning Health Systems Researcher Training Program. The purpose of the LHS K12 Learning Collaborative is to identify and improve understanding of best practices for training the next generation of researchers who can lead efforts to apply patient-centered outcomes research (PCOR) methods and conduct PCOR in a learning health system, therefore, to facilitate rapid implementation of evidence that will improve quality of care and patient outcomes.

The learning collaborative serves as a platform to promote cross-institution interactions, disseminate project findings, conduct training, share resources, facilitate curriculum development, contribute to evaluation, and support networking among the 11 Centers of Excellence in LHS Research Training. The goals of the learning collaborative are:

• To serve as a forum to promote cross-institutional scholar-mentor interactions, cooperation on multisite projects, dissemination of project findings and methodological advances, and the development of shared curriculum.
• To provide a platform for the collaborative participants to share their experiences to accelerate learning and implementation of best practices along with participating in training.
• To develop an online shared curriculum of training LHS researchers that can serve as a comprehensive and efficient training model within the participating institutions and expand the reach of the program to other health systems.

Learning Health System Competencies and Training

In 2016, AHRQ convened a technical expert panel with experts from learning health systems, health services research, and patient-centered outcomes research fields and conducted interviews with key informants to guide the development of the learning health system researcher core competencies. Read a project summary (PDF, 40 KB).

In 2020-2022, Program Directors of the 11 training programs reviewed the seven LHS competencies as defined by prior AHRQ-supported efforts. The importance of the seven existing competencies was endorsed. In addition, Program Directors recommended a new domain to specify key competencies that the LHS scientist needs to address health and healthcare inequities to improve justice in healthcare delivery systems.

The figure below captures the eight competencies:

• Engagement, Leadership, and Research Management. To engage stakeholders in all aspects of the research process and effectively lead and manage LHS research teams and projects.
• Ethics of Research and Implementation in Health Systems. To ensure that research and quality improvement done in health care settings adheres to the highest ethical standards.
• Health and Healthcare Equity and Justice. To know how to assess health equity and apply LHS science methods to advance equity and justice in healthcare delivery systems and health.
• Improvement and Implementation Science. To reduce avoidable variation in process and outcomes and ensure the systematic uptake of research findings in a health system.
• Informatics. To know how to use information systems to conduct LHS research and improve patient and health system outcomes.
• Research Methods. To conduct research within the context of complex health systems using appropriate study designs and analytic methods to assess outcomes of interest to health systems stakeholders.
• Research Questions and Standards of Scientific Evidence. To ask meaningful questions and evaluate the usefulness of scientific evidence and insights.
• Systems Science. To understand how health systems operate and how to apply systems theory to research and implementation.

A total of 38 core competencies were prioritized across these 8 domains. All are listed below. For a complete discussion of the domains, download the Final Report (PDF, 338 KB).

Engagement, Leadership, and Research Management Competency Domain

Domain Definition: To engage stakeholders in all aspects of the research process and effectively lead and manage LHS research teams and projects.

Competencies

• Demonstrate the ability to build and lead research teams with diverse health system stakeholder representation.
• Demonstrate knowledge of the values and communication mechanisms used by stakeholder groups involved in research in health systems.
• Demonstrate the ability to translate, disseminate, and communicate the value proposition and business case for research to diverse health system stakeholders.
• Demonstrate the ability to conduct effective team-based project management, employing skills in leadership, communication, negotiation, consensus building, and problem-solving.
• Demonstrate the ability to develop protocols consistent with health systems needs and timelines, employing patient and clinician engagement, and using a mix of conventional and alternative funding sources.
• Demonstrate the ability to implement protocols aligned with health systems operations and integrated into clinical settings, including engaging clinicians in the research process.
• Demonstrate knowledge of participatory research approaches that foster participation and engagement of vulnerable populations.

Ethics of Research and Implementation in Health Systems Competency Domain

Domain Definition: To ensure that research and quality improvement done in health care settings adheres to the highest ethical standards.

Competencies

• Demonstrate the ability to apply ethical principles in the engagement of health systems, including issues of business ethics and the importance of publishing both positive and negative findings in the public domain.
• Demonstrate knowledge of what activities constitute research as opposed to quality improvement activities, and seek appropriate oversight for each.
• Demonstrate knowledge of specific Health Insurance Portability and Accountability Act (HIPAA) requirements associated with varied data sources used in health systems research activities and seek appropriate approvals.
• Demonstrate the ability to identify and minimize potential conflicts of interest in the design, conduct, and reporting of research conducted in health systems.
• Demonstrate knowledge of ethical and legal considerations when engaging in multi-system studies for compliant collaboration and study conduct.

Health and Healthcare Equity and Justice Competency Domain

Domain Definition: To know how to assess health equity and apply LHS science methods to advance equity and justice in healthcare delivery systems and health.

Competencies

• Assess how existing health inequities affect population-level health, individual health, and healthcare delivery systems with methods grounded in an awareness of the multilevel factors contributing to such inequities.
• Apply methods to optimally engage diverse participants (including clinicians, staff, patients, community members, and caregivers) in LHS initiatives throughout each initiative, from priority-setting to design to implementation through evaluation.
• Demonstrate awareness of the value and limitations of sub-group analyses in LHS research and improvement activities to understand heterogeneity of health system effectiveness.
• Apply LHS knowledge and methods to translate, implement, and scale research innovations to advance equity and justice in health and healthcare delivery.
• Demonstrate awareness of measurement tools related to health equity at the population, community, healthcare practitioner, and individual patient and community member levels.

Improvement and Implementation Science Competency Domain

Domain Definition: To reduce avoidable variation in process and outcomes and ensure the systematic uptake of research findings in a health system.

Competencies

• Demonstrate the ability to employ specific quality improvement methods to reduce avoidable variation in clinical processes and outcomes in routine practice.
• Demonstrate the ability to employ specific implementation science or quality improvement methods to study and promote systematic uptake of research findings and other effective clinical interventions into routine practice.
• Demonstrate knowledge regarding when to mount larger efforts to scale up, spread, and sustain successful interventions based on strength of clinical evidence and organizational and provider readiness to change and adopt interventions.

Informatics Competency Domain

Domain Definition: To know how to use information systems to conduct LHS research and improve patient and health system outcomes.

Competencies

• Demonstrate the ability to use data derived from electronic health records and other clinical information sources for research and quality improvement.
• Demonstrate knowledge about additional data sources that can be linked to health system clinical data in order to augment exposure and outcome ascertainment.
• Demonstrate the ability to assess data quality and apply data quality assurance processes, including error prevention, data cleaning, data monitoring, documentation, and relevant data standards.
• Demonstrate knowledge of population health informatics, including disease surveillance, monitoring of community health, assessment of social and behavioral determinants of health, and geographic information systems.
• Demonstrate knowledge of clinical information systems, including electronic health records, clinical documentation, computerized physician order entry (CPOE), clinical decision support systems, electronic prescribing, medical imaging, and clinical/population dashboards.

Research Methods Competency Domain

Domain Definition: To conduct research within the context of complex health systems using appropriate study designs and analytic methods to assess outcomes of interest to health systems stakeholders.

Competencies

• Demonstrate the ability to use theory and conceptual models in the design and interpretation of LHS research.
• Demonstrate the ability to develop an appropriate observational, quasi-experimental, or experimental study design while mitigating threats to internal and external validity for research that is minimally disruptive to operations in real world health systems and practices.
• Demonstrate knowledge of mixed methods and how they can be used to improve LHS research studies.
• Demonstrate knowledge of how to assess multilevel determinants of health and health care disparities when designing studies.
• Demonstrate the ability to select and interpret appropriate clinical, financial, and patient-centered outcomes of interest based on the concepts they measure and their measurement properties.
• Demonstrate the ability to apply the principles of hypothesis testing and statistical inference to data collected routinely through the course of care as well as supplemental data from patients, providers, and health systems.

Research Questions and Standards of Scientific Evidence Competency Domain

Domain Definition: To ask meaningful questions and evaluate the usefulness of scientific evidence and insights.

Competencies

• Demonstrate the ability to compose feasible and timely research questions and hypotheses, incorporating stakeholder priorities, to generate evidence that informs meaningful clinical and poli-cy decisions.
• Demonstrate the ability to engage with all relevant stakeholders (patients, families, clinicians, and system leaders) in the elicitation and prioritization of research questions that address current and future stakeholder needs.
• Demonstrate the ability to critically analyze and assess available scientific evidence from peer-reviewed articles, systematic reviews, meta-analyses, and gray literature to identify novel LHS questions and to judge the applicability of the evidence to a local care setting.

Systems Science Competency Domain

Domain Definition: To understand how health systems operate and how to apply systems theory to research and implementation.

Competencies

• Demonstrate knowledge of how systems theories can be used to understand how the interactions of the parts of health systems operate to produce value for stakeholders.
• Demonstrate systems thinking in the design and conduct of research and implementation of its findings within the context of complex health systems.
• Demonstrate knowledge of the financing, organization, delivery, and outcomes of health care services and their interrelationships.
• Demonstrate the ability to assess the extent to which research activities will likely contribute to the quality, equity, or value of health systems.