ROUTINE HEALTH INFORMATION SYSTEMS

A Curriculum on Basic Concepts and Practice

MODULE 2:
Indicators and Data Collection and Reporting

SESSION 1:
Indicators
The complete RHIS curriculum is available here:
https://www.measureevaluation.org/our-work/
routine-health-information-systems/rhis-curriculum
SESSION 1 : Learning Objectives

Participants will be able to:

•Define and identify relevant health indicators
•Mention 5 characteristics of good indicators
•Identify factors to consider while selecting
indicators
•Give examples of good indicators
What Are Health Indicators?

With the goal of good health in mind,

think of an indicator as
“…a measure that helps quantify the
achievement of a goal.”
―Mark Friedman
An Indicator Is …….

• a variable (its value changes)…

• that measures (objective calculation of value)…

• key elements of a health intervention (program,

service, or project)
o (inputs/processes, outputs, outcomes, impact)
 Types of Health Indicators

INPUTS AND OUTPUTS OUTCOMES IMPACT

PROCESSES

• Availability of • Number of • Full • Life expectancy

drugs per care children immunization at birth
level immunized rate • Infant mortality
• Basic • Number of interventions • Prevalence of
equipment per new cases • Incidence malnourished
activity type in the and children under 5
• Number of facilities prevalence of
staff per • Hospital certain
category and bed diseases, such
per capita occupation as HIV,
• Number of rate malaria, TB
drug shortages • Number of TB
• Number of cases
surgical or detected
obstetrical
acts done

Source: Adapted from WHO. Global reference list of 100 core health indicators. Retrieved from
http://www.who.int/healthinfo/indicators/2015/en/.
 EXAMPLE: Indicators for Reproductive,
Maternal, and Child Health Monitoring
INPUTS AND OUTPUTS OUTCOMES IMPACT
PROCESSES

•Health financing •Service access •Coverage of interventions •Health status

•expenditure per and readiness •antenatal care
target population •facilities that offer •births attended by skilled health •under-5 mortality
(children, women, and meet tracer personnel •maternal mortality
etc.) criteria for basic •immunization coverage ratio
•General and •family planning needs satisfied •child mortality by
government comprehensive major cause of
obstetric care, •children with diarrhea receiving oral
•expenditure on rehydration therapy death, by sex and
per 10 000 age
health as a pregnant women •children with fever receiving antimalarials
percentage of •insecticide-treated bednet use
general •Financial risk
government •caesarean •antiretroviral prophylaxis among HIV-
section rate in positive pregnant women protection
expenditure
rural populations •vitamin A supplementation among •out-of-pocket
•Health workforce children payments as a
•midwives, per 10 percentage of
•facilities that offer •postnatal care total health
000 population and meet tracer expenditure
•Governance criteria for child
health services, •Risk factors and behaviours
•presence of key
policies to promote per 1000 children •contraceptive prevalence
maternal and child •access to safe water
health •access to improved sanitation
•Information •low birth weight among newborns
•births registered •early initiation of breastfeeding
•deaths registered •children who are stunted or underweight
(with cause)

Source: Adapted from Monitoring, evaluation and review of national health strategies: a country-led platform for information and
accountability. Geneva, World Health Organization, 2011.
 Indicator Domains

• Health status
• Risk factors
• Service coverage
• Health systems

Source: Adapted from WHO. Global reference list of 100 core health indicators Retrieved from
http://www.who.int/healthinfo/facility_information_systems/en/
What Is the Rationale for Health Indicators?

Indicators are vital in health interventions

because, when collected and used regularly,
they can:
•Provide a reference point for health intervention
planning, management, and reporting

•Allow managers of health interventions to assess

trends and identify problems

•Act as early warning signals for corrective action

What Makes an Indicator “SMART” ?
(Characteristics of Good Indicators)

• Specific: Indicator is concrete, detailed, focused, and

well defined
• Measurable: Indicator tells how many or how much
and can be measured with identified
measurement sources
• Agreed upon: Stakeholders vested in a specific M&E
question should agree that the indicator is relevant
• Relevant: Indicator generates data that can answer
the question of interest
• Timebound: Indicator specifies time frame of what it is
measuring
Characteristics: Relevant

• Answers the question of interest: too many

indicators have been defined without being
based upon a specific information need
• Linked to a public-health impact or to
achieving the objectives needed for impact
Example: Childhood vaccines program
Indicator: % of infants receiving measles vaccine
(coverage)
Example: Program to increase access to ORS for
childhood diarrhea through community-based
distributors (CBDs)
Indicator: # of ORS packets distributed in past month
by CBDs
Characteristics: Specific

Or “valid” = measures a specific disease,

service provided, practice, or task
• Indicator measures what it is intended to
measure: provides direct information about
the result it intends to measure
• If indicator not “feasible,” sometimes need
to use “proxy indicator”
Specificity (or Validity): Class Activity

• Is the number of antenatal care (ANC) visits a

valid indicator of a focused ANC package?
• Is the maternal mortality ratio a valid
indicator of the impact of a family planning
program on women’s health?
• Is the number of children who received the
DTP3 vaccine a valid indicator of
immunization coverage?
Characteristics: Measurable

• Quantifiable: using available tools and

methods
• Precise: operationally defined in clear terms
and documented in an Indicator Reference
Sheet
• Reliable: consistently measurable in the same
way by different observers
• Feasible: the resources (human, physical,
financial) needed are available
Indicator Reference Sheet: Minimum
Information

• Description
• Plan for data collection
• Plan for data analysis
• Plan for data quality check
• Performance table
Characteristics: Time Bound

Provides a measurement over periods

of interest, with data available for all
appropriate intervals

Timeliness considerations:
• Reporting schedules
• Recall periods
• Length of time over which change can
be detected
Characteristics: Agreed Upon

• Stakeholders vested in a specific M&E

question should agree that the indicator
is relevant
• Agreement between various health-system
levels
• Agreement between various national health
programs and health services managers
Common Indicator Metrics

Counts
# of providers trained
# of condoms distributed
Calculations: percentages, rates, ratios
% of facilities with trained provider
Index, composite measures
Index on infection control and prevention
DALY (disability-adjusted life years)
Thresholds
Presence, absence
Predetermined level or standard
Factors to Consider When Selecting Indicators

 Logic/link to framework
 Programmatic needs/information for
decision making
 Resources
 External requirements (government, donor,
headquarters)
 Data availability
 Standardized indicators
 Alignment with national and international
standards
Common Pitfalls in Indicator Selection

• Indicators not linked to program activities

• Poorly defined indicators
• Indicators that cannot realistically be collected
• Process indicators to measure outcomes and
impacts
• Indicators that are insensitive to change
• Too many indicators
Pitfalls in Selecting Indicators

• Indicators not linked to program activities

• Intermediate result (IR): Expanded access to malaria

treatment services

• Activities: Train providers in current clinical protocols

• Inappropriate indicator: % of facilities with

adequate conditions to provide care

• Better indicators: # of clinicians trained; % of

facilities with a trained provider

• The program is not aiming to affect facility conditions,

only provider skills.
Pitfalls in Selecting Indicators

• Data needed for indicator not available

• Inappropriate indicator: % of days per quarter that
service delivery points have stockout of drugs

• Data issue: Information on stockouts may not be

collected daily

• Better indicator: % of service delivery points that had a

stockout of drugs at some time during the past quarter

• If relying on routine data, an indicator definition

must depend on how data are collected (see
Session 2) in this module.
Pitfalls in Selecting Indicators

Indicator does not accurately represent desired outcome

• IR: Access to effective treatment among children <5 years old with
malaria

• Inappropriate indicators: % of children <5 years old who

received artemisinin-based combination therapies (ACTs); % of
people who received ACTs for malaria infection who are
children <5

• Better indicator: % of children <5 years old who were diagnosed

with malaria in the past 2 weeks who received ACTs

• What does it mean if inappropriate indicators increase? Decrease?

Do they reflect the desired program effect?
Indicator Systems: How Much Is Enough?

Rule of Thumb
•At least one or two indicators per key activity or result
(ideally, from different data sources)
•At least one indicator for every core activity (such as
distribution of insecticide-treated nets, indoor residual
spraying, training, behavior change communication)
•No more than 8–10 indicators per area of significant
program focus
•A mix of data collection strategies/sources
Choosing the Right Number of Indicators
Good Indicators

• Provide information useful for program decision

making
• Are consistent with international standards and
other reporting requirements, as appropriate
• Are defined in clear and unambiguous terms
• Are nondirectional; “independent”
• Have values that are:
o Easy to interpret and explain
o Precise, valid, and reliable measures
o Comparable across relevant population
groups, geography, and other program
factors, as needed
Change in Indicators over Time
Activity: Handout 2.1.1

1. Share Handout 2.1.1 on change in

indicators over time.

2. Participants read the text individually

3. Discuss in the large group.

“Not everything that can be
counted counts, and not everything
that counts can be counted.”

― Albert Einstein
 Summary: Guiding Principles for
Selecting Indicators

• Ensure that the indicators are linked to the

information needs for health interventions
and are able to measure change.
• Ensure that standard indicators are used to
the extent possible.
• Consider the cost and feasibility of data
collection and analysis.
• Keep the number of indicators to the
minimum that are necessary, and include
only those needed for program and
management decisions or for reporting.
Q&A
 ROUTINE HEALTH INFORMATION SYSTEMS
A Curriculum on Basic Concepts and Practice

This presentation was produced with the support of the United States Agency for
International Development (USAID) under the terms of MEASURE Evaluation
cooperative agreement AID-OAA-L-14-00004. MEASURE Evaluation is implemented by
the Carolina Population Center, University of North Carolina at Chapel Hill in
partnership with ICF International; John Snow, Inc.; Management Sciences for Health;
Palladium; and Tulane University. The views expressed in this presentation do not
necessarily reflect the views of USAID or the United States government.