UNIT No.

4
DESINING A MONITERING & EVALAUTING PROCESS

There are many methods and tools that can be used for monitoring. In choosing and using
these, it is important to ensure the information produced is of good quality, useful for the
monitoring purpose and consistent over time. These requirements should govern the way you
collect and manage your data.
Collecting and Analyzing Information

 When you collect data you should be confident that people want to and will be able to use
and understand it. Much effort has been expended collecting data that no one looks at. Think
about who will use it: who stands to gain most from it? Whose behaviors might change as a
result of the lessons learnt from it? This will lead you to consider what level of detail you
need to record.
 Regulations may prescribe the need for some types of information, the methods used to
collect information, the way in which it is presented, and the timing of its presentation. These
requirements need to be factored into an overall methodology for data collection.
 Data collection can be made more efficient by enlisting the help of people who are working
in the field (eg, consent planners, building officers, technicians). If you make it easy for
people to collect and record data as they are doing their jobs, it helps with consistency and
reduces the chances of getting gaps in the information. It is a good idea to get input on design
of data collection forms from the people who will collect the data.
 Find out what information is collected by other agencies that you can use. There may also be
opportunities to share the costs of your data collection with other agencies who also want the
information.
 Think about how often you need to collect the data to provide information about trends for
reporting and review. If change in the values of a particular indicator will only be apparent
over several years, you might not need to measure it annually.

Managing data and information

 Just collecting data is not enough - to have confidence in its quality, consistency, and
accessibility over time means developing clear procedures for its measurement, recording,
and storage and security as well as descriptions about how these things change. To ensure it
will be accessible for future users, it must also be stored and described in suitable ways.

1
 Metadata is information about the data - such as what, how, where, when, how often and by
whom data is collected, as well as how it is recorded, stored and analysed. This information
allows you to determine whether or not data sets collected at different times are truly
compatible and so able to be combined to build accurate time series. It preserves a clear
understanding of the data over the years despite staff turnover. It can set out criteria for
significance when interpreting the data, as well as make clear any limitations in the
measuring techniques applied or samples taken.
 Metadata acts like a library catalogue. It describes topics, how they can be accessed, and
where to direct enquiries. In this way it assists the public, other agencies, and your own
colleagues to locate all available data in a field of interest. This helps to prevent duplication
of effort and to share knowledge. Shared knowledge can translate to shared commitment. An
effective approach is to have your metadata on the internet for ease of access.
 Key matters to consider in designing a system to manage data are:
 What is required by regulation?
 How would a person find relevant data?
 What makes the data fit for purpose?
 How will the data be shared and with whom?
 How will it be secured for future use?
 Consider whether there is any opportunity to link or combine your council databases to
include all monitoring information relevant to state of environment, policy/ plan effectiveness
and compliance and complaints monitoring. Integrated data storage can occur at different
levels - metadata, indicators, and underlying data.
 Consider having a data review process as quality of data is critical

DATA COLLECTION TECHNIQUES

Information you gather can come from a range of sources. Likewise, there are a variety of
techniques to use when gathering primary data. Listed below are some of the most common
data collection techniques.

 Interviews
 Questionnaires and Surveys
 Observations
 Focus Groups
 Ethnographies, Oral History, and Case Studies
 Documents and Records

2
OVERVIEW OF DIFFERENT DATA COLLECTION TECHNIQUES
Technique Key Facts Example
 Interviews can be conducted
in person or over the
telephone
 Interviews can be done
One-on-one conversation with parent
formally (structured), semi-
of at-risk youth who can help you
structured, or informally
Interviews understand the issue
 Questions should be focused,
clear, and encourage open-
ended responses
 Interviews are mainly
qualitative in nature

 Responses can be analyzed

with quantitative methods by
Results of a satisfaction survey or
assigning numerical values to
opinion survey
Likert-type scales
Questionnaires  Results are generally easier
CYFERnetSEARCH
and Surveys (than qualitative techniques)
Interactive Survey Builder feature.
to analyze
 Pretest/Posttest can be
compared and analyzed

 Allows for the study of the

dynamics of a situation,
frequency counts of target
behaviors, or other behaviors
as indicated by needs of the
evaluation
 Good source for providing
additional information about
a particular group, can use Site visits to an after-school program
Observations video to provide to document the interaction between
documentation youth and staff within the program
 Can produce qualitative (e.g.,
narrative data) and
quantitative data (e.g.,
frequency counts, mean
length of interactions, and
instructional time)

3
  A facilitated group interview
with individuals that have
something in common
A group of parents of teenagers in an
 Gathers information about
after-school program are invited to
combined perspectives and
Focus Groups informally discuss programs that
opinions
might benefit and help their children
 Responses are often coded
succeed
into categories and analyzed
thematically

 Involves studying a single

phenomenon
 Examines people in their
natural settings
Shadowing a family while recording
 Uses a combination of
extensive field notes to study the
techniques such as
Ethnographies, experience and issues associated
observation, interviews, and
Oral History, and with youth who have a parent or
surveys
Case Studies guardian that has been deployed
 Ethnography is a more
holistic approach to
evaluation
 Researcher can become a
confounding variable

 Consists of examining
existing data in the form of
databases, meeting minutes,
To understand the primary reasons
reports, attendance logs,
students miss school, records on
financial records, newsletters,
Documents and student absences are collected and
etc.
Records analyzed
 This can be an inexpensive
way to gather information but
may be an incomplete data
source

Decision making
In psychology, decision-making is regarded as the cognitive process resulting in the
selection of a belief or a course of action among several alternative possibilities. Every
decision-making process produces a final choice, which may or may not prompt action.
Decision-making is the process of identifying and choosing alternatives based on
the values, preferences and beliefs of the decision-maker.
Characteristics of decision-making

4
 Objectives must first be established
 Objectives must be classified and placed in order of importance
 Alternative actions must be developed
 The alternatives must be evaluated against all the objectives
 The alternative that is able to achieve all the objectives is the tentative decision
 The tentative decision is evaluated for more possible consequences
 The decisive actions are taken, and additional actions are taken to prevent any adverse
consequences from becoming problems and starting both systems (problem analysis and
decision-making) all over again
 There are steps that are generally followed that result in a decision model that can be used
to determine an optimal production plan[7]
 In a situation featuring conflict, role-playing may be helpful for predicting decisions to be
made by involved parties[8]

Decision-making techniques

Decision-making techniques can be separated into two broad categories: group decision-
making techniques and individual decision-making techniques. Individual decision-making
techniques can also often be applied by a group. Group

 Consensus decision-making tries to avoid "winners" and "losers". Consensus requires that
a majority approve a given course of action, but that the minority agree to go along with
the course of action. In other words, if the minority opposes the course of action,
consensus requires that the course of action be modified to remove objectionable
features.
 Voting-based methods:
 Majority requires support from more than 50% of the members of the group. Thus,
the bar for action is lower than with consensus.
 Plurality, where the largest block in a group decides, even if it falls short of a
majority.
 Range voting lets each member score one or more of the available options. The
option with the highest average is chosen. This method has experimentally been
shown to produce the lowest Bayesian regret among common voting methods, even
when voters are strategic.
 Delphi method is a structured communication technique for groups, originally developed
for collaborative forecasting but has also been used for policy making.
 Dotmocracy is a facilitation method that relies on the use of special forms called
Dotmocracy Sheets to allow large groups to collectively brainstorm and recognize
agreement on an unlimited number of ideas they have authored.
 Participative decision-making occurs when an authority opens up the decision-making
process to a group of people for a collaborative effort.
 Decision engineering uses a visual map of the decision-making process based on system
dynamics and can be automated through a decision modeling tool, integrating big
data, machine learning, and expert knowledge as appropriate.

5
Individual

 Decisional balance sheet: listing the advantages and disadvantages (benefits and costs,
pros and cons) of each option, as suggested by Plato's Protagoras and by Benjamin
Franklin.[12]
 Simple prioritization: choosing the alternative with the highest probability-
weighted utility. This may involve considering the opportunity cost of different
alternatives. See also Decision analysis.
 Satisficing: examining alternatives only until the first acceptable one is found. The
opposite is maximizing or optimizing, in which many or all alternatives are examined in
order to find the best option.
 Acquiesce to a person in authority or an "expert"; "just following orders".
 Anti-authoritarianism: taking the most opposite action compared to the advice of
mistrusted authorities.
 Flipism e.g. flipping a coin, cutting a deck of playing cards, and other random or
coincidence methods – or prayer, tarot cards, astrology, augurs, revelation, or other forms
of divination, superstition or pseudoscience.
 Automated decision support: setting up criteria for automated decisions.
 Decision support systems: using decision-making software when faced with highly
complex decisions or when considering many stakeholders, categories, or other factors
that affect decisions.

JUDGING
It is the process of giving remarks after thorough analysis of data and factors. It is termed as
evaluation process in terms of educational programs.
are professional groups that review the quality and rigor
of evaluation processes. Evaluating programs and projects, regarding their value and impact
within
REPORTING

Reporting may refer to any activity that leads to reports in particular

business reporting Data reporting Sustainability reporting, Financial reporting
Reporting may refer to

 any activity that leads to reports

 in particular business reporting

 Data reporting
 Sustainability reporting
 Financial reporting
 international reporting of financial information for tax purposes under the
OECD's Common Reporting Standard
 Journalism
 Court reporting
 Traffic reporting

6
 Beat reporting
 Operational reporting