RBME

Download as ppt, pdf, or txt
Download as ppt, pdf, or txt
You are on page 1of 72

Training Workshop on Project Planning and

Management and Monitoring and Evaluation

7th - 13th December, 2020


Adama
RESULT BASED MONITORING
&
EVALUATION
(RBME)
Training Objectives
 At the end of the session
participants will;

 get clear insight in the


conceptual definition, objectives
and use of RBME;

 have knowledge and skill on how


to set-up system of M & E and
practically apply it in the
context of Agricultural Projects

06/19/23 GIRMA KEBEDE 2


Reflection Exercise 1
(15 minutes)
• What major agricultural investment programs and
projects are currently on progress in your
zone/institution?

• What is your overall observation/judgment in their


progress/performance?

• Have you ever attempted to engage in M&E of the


projects you mentioned? If yes what have you
achieved or what challenges have you faced?
Introduction
• Growing pressures on governments and
organizations for:
– good governance,
– accountability and transparency,
– greater development effectiveness, and
– delivery of tangible results.
• Result Based Monitoring and Evaluation
(RBME) including control is a powerful
management tool that can be used to
improve the way governments, all
development actors and organizations
achieve results.

• It help policy and decision makers and


implementers to track progress and
demonstrate the impact of a policy,
strategies, programs and projects
implemented on citizens
• M & E provide:
– macro and micro-level description and
analysis, respectively of the process and
effects of certain policy or intervention
(program/project) to policy makers, planners,
managers and those who allocate the
necessary resources and made ultimate
decisions
• Despite the strong belief on the importance of M&E
as a tool to be used in management at all levels,
there is a gap or little awareness in terms of
understanding the concept of the tool and its
application.

• Particularly the deficiency in designing and/or using


effective Monitoring & Evaluation (M&E)
system/mechanism which is geared at tackling with
implementation constraints is often evident in many
organizations (particularly in public organizations).

• The need to develop and apply practical M&E system


is increasingly recognized by governments in many
developed and developing countries
What is
Monitoring & Evaluation?
and
Why, When & How we
M&E?
Development Project
Cycle
Impact 1
Problems &
Potentials
Identification

Formulation
6 Evaluation
2

Operation Monitoring & Evaluation

Implementation Appraisal
5
3
Approval

4
• Monitoring and Evaluation are related
but distinct sets of organizational
activities.
What is Monitoring ?
• Monitoring is a continuous process or activity that
involves;
– collecting
– recording
– communicating, analyzing and using information for the
purpose of management control and informed decision
making.

• Primarily, monitoring provides the information by


which the concerned body or the management can
assess progress in relation to what was intended
(planned) and identify and solve problems
encountered during implementation.
• From the perspective of MoA/Regional/zonal Agriculture
Offices monitoring can be described as:
– systematic collection of data on specified indicators of an
approved and implemented
• agricultural investment plans /programs and projects/
by the Ministry or its regional offices or investment
agencies accountable to it with indications of the extent of
progress, achievement of objectives and in the use of
allocated resources (finance, technology, land and other
supports provided) in order to make the necessary
decisions and also provide feed back and directions for
the project implementing bodies .
What is Evaluation?
• … is systematic and periodic objective
assessment of an ongoing or completed plan,
program, project or policy, including its design,
implementation, and results.
• .. the aim is to determine the
– relevance and fulfillment of objectives,
– development efficiency,
– effectiveness,
– impact, and
– sustainability.
• An evaluation should provide information that is
credible and useful, enabling the incorporation of
lessons learned into the decision making process.
Monitoring Vs Control
• In the context of management, control is:
– is the comparison of what has been achieved with
what was planned or decided
– is verifying whether everything occurs in
conformity with the policies issued, plans adopted,
instructions issued and principles/rules
established as well as determination of necessary
adjustments required to the plan or decisions
under implementation in order to mitigate the
effects, on the overall performance and
achievements of result.
06/19/23 GIRMA KEBEDE 14
Why M&E?
&
Who requires it?
Do you think that M&E is
a government or funding/
sponsoring agencies
requirement?

06/19/23 GIRMA KEBEDE 16


 It is a management tool for any organization
which helps to ;
 assess how far we are doing against objectives/review
progress,
 see whether we are achieving our intended result or not,
 check whether we are working efficiently
 learn how to do it better (learning and evolving)
• review progress;
• have clearer basis for decision making
• develop accountability
• empower stakeholders
• promote learning and knowledge
• identify problems in planning and/or implementation; and
• make adjustments so that we can more likely to “make a
difference”.
06/19/23 GIRMA KEBEDE 17
• M&E is not a solution, but it is a valuable tool
that can:
– help to identify problems and their causes;
– suggest possible solutions to problems;
– raise questions about assumptions and strategy;
– push us to reflect on where we are going and how we
are getting there;
– provide us with information and insight;
– encourage us to act on the information and insight;
– increase the likelihood that we will make a positive
development difference
Please note that we may not
give equal emphasis for M&E
conducted during planning
and implementation/operation/
stages.
WHY M&E is Important During Implementation of
Policies, Plans, Programs & Projects?
 Whatever concentration is made on planning,
inadequately or poorly managed policies and
programs/projects during implementation will
fail or be expensive to realize

 During implementation management is


subject to many influences which can be
hardly foreseen during the design/planning
stage.
 Any plan (project or program plans or policy
documents) can never be assumed perfect and
corrective measures are always expected
during the course of implementation
06/19/23 GIRMA KEBEDE 20
WHY M&E …

Particularly
– objectives …
– planning assumptions…
– implementation methods …
– the context, etc.
must be continuously questioned and checked
on the basis of implementation experience and
changes observed (internal and external).
 Management at all levels needs continuous
flow of information on these changes in order
to be able to successfully mange the
implementation.

06/19/23 GIRMA KEBEDE 21


WHY M&E …

 The best management instrument for


obtaining adequate flow of information is
establishment of effective M&E system.

 Hence, adequate attention should be paid to


design and use practicable M&E system to
ensure effective implementation of our
policies, plans, programs ...

06/19/23 GIRMA KEBEDE 22


In general:
– M&E is an important tool for those who manage
anything from a small project component to an
entire country.

– Monitoring gives information on where a policy,


program, or project is at any given time (and over
time) relative to respective targets/outputs and
outcomes. It is informative in intent.

– Evaluation gives evidence of why targets/outputs


and outcomes are or are not being achieved. It
seeks to address issues of causality. It is
descriptive in intent.
• Effective M&E system leads an organization
to be SMART on its governance i.e.

S: Simplifying its operation and services

M: Moral based

A: Accountable

R: Responsive /Citizen Focused

T: Transparent
Assess Assess
Goals, Progress of
Objectives Activities Determine whether
& Strategies implementation is
according to schedule

Mobilize
Stakeholders, Enhance
Teamwork & Build
Shared Commitment

Assess Results

Plan Multiple Attained

Improvement
Purposes of
M&E Assess as the
targeted groups and
Practice Bench- area are reached
Marking

Identify Lessons for


change and
Enhance Improvement
Accountability

Gather information
for early warning
Forecast
Ensure Quality Performance
Management
25
Group Discussion
(20 minutes)

• From the previous discussion and your own


practical experience as a professional staff of
Regional/Zone Agri. Office describe the
distinction of monitoring and evaluation
based on the following characteristics.
• Purpose/objective (What does it answers?)
• Actors/Responsible Body to undertake M&E/
• Primary Users of M&E Results
• Frequency
• Type of Data Gathered
• Focus
• Main Action
Distinctive Characteristics
of M & E
Characteri Monitoring Evaluation
stics
Purpose/objective - Activities performed - Why and how
(What does it - Outputs achieved results were achieved or not
answers?) -Resources used -Strategy and policy options

-Quality of work performed

- Problems encountered (focus on effectiveness,


-Rectifying measures efficiency, relevance, impact,
etc.)
(focuses on inputs, process,
output and work plan)
Actors Mainly Internal Internal/External
Primary Users Small group/Managers Large group /Managers,
Planners, Financers,
decision Makers etc.)
27
Distinctive Characteristics …
Characteristics
Monitoring Evaluation
Frequency/Time Continuous Periodic
Type of Data Primarily Primarily
Gathered Quantitative Qualitative
Main Action Oversight and Timely In-depth analysis
Corrective Action and acquire lesson
for future action
Focus Inputs/Outputs/ Outcome/Impact
Outcome and Sustainability

28
Common Features of M&E
 In many cases both M&E use the same data collection and
analysis system

 Most of the indicators for monitoring may be included in the


range of information required for evaluation.

 Both provide clear basis for decision making

 Enhance accountability

 Focus on performance & achievement of result

 Empower stakeholders if carried in a participatory way


06/19/23 GIRMA KEBEDE 29
Participatory M & E
• It is a process in which primary and
other stakeholders collaborate and
take an active part in monitoring &
evaluating process

06/19/23 GIRMA KEBEDE 30


Significant Features of Participatory
M&E are:
Focus on process & outcomes
Uses multiple approaches
Encourages diverse views
Creates a learning experience
Involves experts as facilitators
Uses relatively simple and flexible design
Stakeholders are key actors for the activity

06/19/23 GIRMA KEBEDE 31


M&E: Whose Responsibility?
• Responsibility is determined depending
on the government and an
organizations’ structure/hierarchy.
Four Levels of M&E
• Federal level by House of Peoples’
Representatives (the Parliament) and MoF and
Planning Commission

• Organizational level by concerned ministries,


agencies and institutions

• Regional/zone levels by concerned bureaus,


agencies and institutions

• Program/project level by the management unit of


the program/project
Exercise 2
(20 min.)

• Who is responsible to monitor and evaluate


implementation of agricultural projects carried out in
your region and weredas?

• As a staff of the Agricultural office at Region/Zone


level, it is believed that as you have been involved in
an M&E activity before. Thus,
– State the purpose of your M&E activities.
– Examine how realistic and relevant they were?
– Identify the level of stakeholders’ participation both in the
process and decision making?
– Indicate the most important lessons you learned from the
M&E process?
Traditional/Implementation –
Focused M&E

Vs
Result-Based M&E
(RBME)
Implementation – Focused M&E
• designed to address compliance—the
“did they do it?” question.
Did they mobilize the needed inputs?
Did they undertake and complete the agreed
activities?
Did they implement the policies issued?
Did they implement the rules and regulations
issued?
Did they deliver the intended outputs (the
products to be produced or services to be
provided)?
Implementation…
• Implementation Focused approach
does not provide policymakers,
managers, and stakeholders with an
understanding of the
success or failure or
the result/achievements of the policies
issued, program and projects implemented
Result – Based M&E
(RBME)
• Over the last decade RBME is found as
the best and power full Management
tool of public sector development
plans.

• It would help to demonstrate


accountability, transparency, and
results.
What is result?
– … change(s) in a state or condition which derive
from a policy, strategy, program or project.

– The result may be intended or unintended,


positive or negative.

– A result can be an output (product, service or unit


result), an outcome (organizational result) or an
impact (Strategic or Functional Objective, or
Global Goal of organizational Members).
What is …

– Results-based monitoring is a continuous


process of collecting and analyzing
information to compare how well a policy,
program or project/intervention is being
implemented against expected results.
Elements of RBM
Managing
Results

Strategic
Performance
Planning
Operational/Work Management
Planning

• Defining outcome,
Detailed outputs & • Monitoring Results
• Formulating Objectives • Reviewing and
• Defining a Strategy activities
• Assigning Reporting Results
• Identifying Indicators • Using Performance
• Setting Targets responsibilities
• Determining resource Information
needs • Integrating Evaluation
• RBME systems are required to address the
“so what” question.
– So what that activities have taken place?
– So what that the outputs from these activities have
been produced?
• RBME systems help to answer the following
questions:
– What are the objectives and goals of the
government/organization/program/project?
– Are they being achieved? or not
– How can achievement be proven?
What do you think are
the following?
• Building an agro-processing industry in five
different locations in Amhara Region
• 20,000 ha. of land cleared and planted in
Metema wereda,
• Trained employees of MoA or Regional
Bureaus,
• Micro and small enterprise groups
established in rural areas,
• Irrigation dam constructed, etc.
KEY FEATURES OF TRADITIONAL
(Implementation Focused) and
RESULT -BASED M & E

ELEMENTS OF ELEMENTS OF RESULT


TRADITIONAL M & E BASED M & E
Baseline data to describe the problem or
Description of the problem or situation before the intervention;
situation before the
intervention;
Indicators are more emphasis on
Indicators are for activities outcomes and impact;
and immediate outputs;
Data collection focuses on Data collection focuses on outputs and
how/whether they contribute towards
inputs, activities and achievement of outcomes and
program/project and organizational
immediate outputs objectives. Managers are being asked to
actively apply the information gained
through monitoring and evaluation to
improve policies, strategies,
programs/projects performance
KEY FEATURES …
ELEMENTS OF ELEMENTS OF RESULT
TRADITIONAL M & E BASED M & E
Systematic reporting on provision More focus on change among
of input and attainment of output stakeholders by indicating the
performance trend over time;
Directly linked to a discrete Systematic reporting with more
intervention (or series of qualitative and quantitative
interventions); information on the progress of
outcomes
Designed to provide information on Captures information on success or
administrative, implementation and failure of the policy/program or
management issues as opposed to project in achieving desired
broader development effectiveness change or impact
issues.
SO HOW CAN WE APPLY RBME
IN CONTEXT OF MoA
RESPONSIBILITIES?
• Establishing clear result chain before actual
implementation of a policy or a plan i.e.
changes expected and to be observed on the
target group, project area, at regional and
national levels upon progress, completion
and post completion of the project. For
example:
– Ensure stronger agricultural investment capacities in the region/country
– Increase non-farm job creation in the rural sector
– Enhance integrated agricultural development (farming & agro-processing)
– Ensure provision of quality agricultural inputs (improved seed, fertilizer,
etc.)
– Create conducive environment for the development of agro industry in the
country/region
Results Chain

activities
(how the specific program carries out its
work)

outputs
R
(goods, products and services
produced by the
program/intervention carried)
E
S
immediate outcomes
U (purpose)
L (the first-level effects of the outputs)

T
s intermediate
outcomes (Purpose)
(the benefits and changes resulting
from the outputs)

end outcomes (impact)


(the final or long-term consequences)
Result Ladder

oj ect
e pr
e g ic
of t
h
t rat ves
he
li fe S
je cti
i nt Ob
e ct ter
pro
j Lat
t he mes
life
o f
tc o
th e O u
l y in
Ear

u ts
tp
Ou

i ti es
ctiv uts
A np
&I
What specific Items to be
Monitored?
• Whenever we are dealing with
development/investment projects, the
following are among the major items that
have to be closely monitored and
checked.
– Physical progress/performance (inputs, activities, and
outputs)
– Financial expenditure
– Quality of services delivered and/or products produced
– Assumption indicated in the plan
– Beneficiary Perception /outcome/
– Management/Administration efficiency
Physical Progress/performance
Monitoring
• Items to be specifically considered are:-
– Results of activities /outputs/ … what the project
produced or services it delivered
– Inputs utilization
– Progress towards objectives /purpose/
– The way the program is managed and style of work
– Changes observed on the targeted population/area
and context
• Method used ----------------- by checking
milestones set in the plan (result framework)
• Questions to be raised are:
– Are the planned physical work as a whole (and its individual
components) are on schedule, ahead of schedule, or behind
schedule?

– If there is a variation; Where did it occur? Why did it occur? Who is


responsible for it?, and What would be its implications?

– What problems were encountered? What actions were taken and by


whom?

– What issues need focus in the future and by whom?

– How was the management efficiency in the organization/project?

– What is the trend of the physical performance? What would be the


likely final cost and completion date of the activities and outputs
and its individual components?
Financial Utilization
/Expenditure/ Monitoring
• Costs incurred to date (the time of reporting): this can be obtained
by summing up costs incurred in accomplishing various activities

• Budgeted costs to date: this can be readily obtained from the cost
projections made at the beginning of the plan period.
 
• Value of work done to date: when costs are measured, an estimate
should be made of the extent of work accomplished. The value of work
done can then be obtained as follows
Budgeted costs X % of work accomplished
•  Cost over – run (under – run) to date: There is cost over-run when
the cost incurred is more than the value of work done and vise versa.
Cost over – run (under – run) is usually expressed in percentage terms
and defined as follows.
Quality Monitoring

• Varies from place to place depending on


the nature of work planned

• It must be based on original specification


or predetermined indicators/criteria
Assumption Monitoring

• Event, a condition or a decision which is necessary


for a success of our policy or plans, but which is
largely or completely beyond the control of an
organization

• Monitor external factors indicated as an assumptions


and risks during the planning phase
Beneficiary /Users/Public
Perception Monitoring
• During the implementation of any policy, strategy, or plan
(program/project) the perception of the public or intended
beneficiaries has to be monitored to understand or to see its
relevance and made timely corrective actions/ amendments.
• Major questions to be raised during monitoring are;
– Who was supposed to benefit from the products/services and inputs?
– Who got access to it?
– How did they react to it?
– How did the intervention affect their life, production, behavior, performance,
etc.?
– To what extent did persons with access understand the available
services/goods/products?
– To what extent did those who saw and try the services or products found as
relevant?
– Did those who tried the services continue using them? etc.
How to Monitor?
(Monitoring Mechanisms)
How to…

• Several mechanisms are used to collect


data and information for monitoring
and evaluation
• Commonly used methods are:
– reports,
– meetings, and
– Visits/observation/
How to…

• The following monitoring techniques are also


used in many organizations to monitor
issues pertaining to their duties and
responsibilities.
• Performance audit report
• Policy and legal act implementation assessment
• Short time oral and written questions for concerned bodies
• On site inspection
• Document investigation
• Media reports, etc.
• Group Work Presentation
EVALUATION

06/19/23 GIRMA KEBEDE 60


• Evaluation is a key management tool for enhancing
the management efficiency
• Why we evaluate?
Among others to;
– see what had been achieved
– measure progress
– be able to improve our monitoring methods
– see where our strengths and weaknesses lie
– criticize our own work
– compare the achievement of specific work with others similar one
– be able to share our experiences
– make our work more effective
– help us to make better plans for the future
– help us see where we are going and if we need to change direction,
61
etc.
When & What do we Evaluate?

When
i. Ex-ante / start-up/ evaluation,
ii. On-going or mid-term /formative/
evaluation,
iii. Terminal /summative/ evaluation; and
iv. Ex-post/impact evaluation.

06/19/23 62
In addition to the above time which is usually
determined during planning/design, evaluation may
be carried when:

– monitoring report indicates an unexpected result which is


positive or negative

– additional information for decision-making is required

– key questions to be resolved are identified during


monitoring process

– major changes occur in critical elements or key assumptions

– the need to extract key lessons learned arises.


06/19/23 63
What ?
Although it varies on the type of evaluation carried
among others we have to assess:

– relevance of the policy, program, project, etc. to the problem


it is intended to solve
– effectiveness and efficiency
– adequacy of resources (availability & utilization)
– attainment of expected results and impacts
– Whether targeted population/groups/individuals or area are
being reached by the project
– sustainability factors
– Replicability & lessons learned for the future.

06/19/23 GIRMA KEBEDE 64


Major Problems of the
M & E Process

• What do you think are the major


problems of M&E in Ethiopia in general
and your specific sector and
organization in particular?
 Inadequate understanding of and
attention to M&E during planning

 Inadequate or lack of M&E unit and


staff. In most cases M & E system is
not either properly established or not
provided adequate attention and
resources where it exists.

 Lack of commitment to monitoring by


organizational leaders, staff and
implementing partners.;
66
• Monitoring is seen as an obligation
imposed from outside,

• Seeing monitoring only as a form of data


collection in the process of writing
reports for higher level government body
or financers

• Lack of clear accountability for failures


and inadequate reward for special efforts
made
• Poor use of participatory and qualitative
M&E methods, due to limited capacity
and inability to see the need for such
information;

• Very few or no internal reviews or on-


going evaluations experience in
organizations, with adjustments initiated
mainly by external body;
 Limited training opportunity for M&E
personnel in offices where the unit
exists.

 Limited information management


practices (poor documentation). Even
when information is available it doesn't
answer the right questions. Frequently
where the system exists it focus only on
quantitative financial aspects and
physical implementation performance
and neglect the results attained
(outcome and impact).
06/19/23 GIRMA KEBEDE 69
 Late arrival of information required for
monitoring.

 Too costly to collect information.

 Disregard of previous M&E findings in


preparation of subsequent plans or
design of new programs/projects.

 High mobility of staff disrupting


continuity of M&E functions.
06/19/23 GIRMA KEBEDE 70
Reflection Question

• As professionals and practitioners


working in the sector, what do you
suggest to solve these problems?
72

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy