Framework For Managing Programme Performance Information
Framework For Managing Programme Performance Information
ISBN : 978-0-621-37152-9
RP : 86/2007
National Treasury
What you should know...
1. INTRODUCTION .......................................................................................................... 1
1.1 WHY IS MEASURING PERFORMANCE IMPORTANT? ............................................ 1
1.2 AIMS OF THE FRAMEWORK ..................................................................................... 1
1.3 POLICY AND LEGAL REQUIREMENTS..................................................................... 2
1.3.1 Constitution ............................................................................................................... 2
1.3.2 Public sector management reform............................................................................ 2
1.3.3 The Government-wide Monitoring and Evaluation System ...................................... 2
1.4 APPLICABILITY OF THE FRAMEWORK .................................................................... 3
1.5 A WORD ON TERMINOLOGY .................................................................................... 3
2. PLANNING, BUDGETING AND REPORTING............................................................. 4
3. KEY PERFORMANCE INFORMATION CONCEPTS .................................................. 6
3.1 INPUTS, ACTIVITIES, OUTPUTS, OUTCOMES AND IMPACTS............................... 6
3.2 PERFORMANCE INDICATORS .................................................................................. 7
3.3 PERFORMANCE TARGETS ....................................................................................... 8
4. DEVELOPING PERFORMANCE INDICATORS .......................................................... 10
5. MANAGING PERFORMANCE INFORMATION .......................................................... 12
5.1 RESPONSIBILITIES .................................................................................................... 12
5.2 INTEGRATED PERFORMANCE INFORMATION STRUCTURES AND SYSTEMS... 12
5.3 MANAGEMENT CAPACITY......................................................................................... 13
6. PUBLISHING PERFORMANCE INFORMATION ........................................................ 14
6.1 ACCOUNTABILITY REPORTS .................................................................................... 14
6.2 INFORMATION TO FACILITATE OVERSIGHT ........................................................... 14
6.3 PROVIDING PUBLIC ACCESS TO GOVERNMENT-HELD INFORMATION.............. 15
6.4 INFORMATION FOR RESEARCH .............................................................................. 16
6.5 INFORMATION ON THE INTERNET .......................................................................... 16
7. ROLES AND RESPONSIBILITIES .............................................................................. 17
7.1 THE PRESIDENCY AND PREMIERS' OFFICES........................................................ 17
7.2 THE NATIONAL TREASURY AND PROVINCIAL TREASURIES ............................... 17
7.3 NATIONAL DEPARTMENTS RESPONSIBLE FOR CONCURRENT FUNCTIONS.... 17
7.4 THE DEPARTMENT OF PUBLIC SERVICE AND ADMINISTRATION ....................... 18
7.5 THE DEPARTMENT OF PROVINCIAL AND LOCAL GOVERNMENT AND
PROVINCIAL DEPARTMENTS OF LOCAL GOVERNMENT...................................... 18
8. CONCLUSION.............................................................................................................. 19
ANNEXURE 1: GLOSSARY ............................................................................................... 20
National Treasury
Chapter 1
INTRODUCTION
The public sector delivers services essential to the well-being and development of the nation. To ensure
that public service delivery is as efficient and economical as possible, all government institutions are
required to formulate strategic plans, allocate resources to the implementation of those plans, and monitor
and report the results. Performance information is essential to focus the attention of the public and
oversight bodies on whether public institutions are delivering value for money, by comparing their
performance against their budgets and service delivery plans, and to alert managers to areas where
corrective action is required.
Performance information also plays a growing role in budget allocations and will increasingly be used to
monitor service delivery. This means the information must be accurate, appropriate and timely.
The most valuable reason for measuring performance is that what gets measured gets done. If an institution
knows that its performance is being monitored, it is more likely to perform the required tasks - and to
perform them well. In addition, the availability of performance information allows managers to pursue
results-based management approaches, such as
The Power of Measuring Results
performance contracts, risk management,
benchmarking and market testing. • If you do not measure results, you can not
tell success from failure
This document outlines key concepts in the design • If you can not see success, you can not
and implementation of management systems to reward it
define, collect, report and use performance • If you can not reward success, you are
information in the public sector. probably rewarding failure
• If you can not see success, you can not
1.2 Aims of the Framework learn from it
• If you can not recognise failure, you can
not correct it
This Framework aims to:
• If you can demonstrate results, you can
• Clarify definitions and standards for
win public support
performance information in support of regular
Adapted from Osborne and Gaebler, 1992, Reinventing Government
audits of such information where appropriate
• Improve integrated structures, systems and
processes required to manage performance information
• Define roles and responsibilities for managing performance information
• Promote accountability and transparency by providing Parliament, provincial legislatures,
municipal councils and the public with timely, accessible and accurate performance information.
performance against predetermined objectives of the auditee, which include constitutional institutions, departments, trading entities, public entities, municipalities and municipal
entities, and other institutions as indicated by sections 4(1) and 4(3) of the act.
Section 92 of the Constitution states that "members of the Cabinet are accountable collectively and
individually to Parliament for the exercise of their powers and the performance of their functions",
and that they must "provide Parliament with full and regular reports concerning matters under their
control". Section 133 provides for the accountability of members of the executive council (MECs)
of a province to the provincial legislature. Similar arrangements are specified for municipalities in
the Municipal Structures Act (1998).
The implementation of the Public Finance Management Act (PFMA) (1999), the Municipal
Finance Management Act (MFMA) (2003) and the Public Service Act (1994 as amended) has
enhanced control over public expenditure and empowered public sector managers. One challenge
for the public sector is to use resources in a more efficient way. Further policy initiatives and legal
requirements have been introduced to achieve this, including the integration of performance
concepts from the Estimates of National Expenditure (ENE) and other budget documents.
In 2004, the Cabinet initiated plans for a monitoring and evaluation system for government, and
the Presidency subsequently developed the Government-wide Monitoring and Evaluation
Framework.
Although there are various existing systems gathering valuable information within government,
there are also a number of gaps in the information needed for planning the delivery of services and
for reviewing and analysing the success of policies.
The Government-wide Monitoring and Evaluation System seeks to enhance these systems by
describing them and explaining how they relate to each other.
The following figure illustrates the relationship between these components. It highlights that there will be
frameworks dealing with each component. The Framework for Managing Programme Performance
Information deals with the management of the programme performance information component, although
the terminology and definitions outlined in it are generally applicable throughout the Government-wide
Monitoring and Evaluation System.
National Treasury 2
Chapter 1: Introduction
The planning, budgeting and reporting cycle describes the relationship between these processes and emphasises
that the executive is accountable to the relevant elected representative body for the entire process. Full and regular
reports are required at each stage of the process.
At any given time within government, information from multiple years is being considered: plans and budgets for
next year; implementation for the current year; and reporting on last year's performance. Although performance
information is reported publicly during the last stage, the performance information process begins when policies
are being developed, and continues through each of the planning, budgeting, implementation and reporting stages.
The documents associated with each stage provide the relevant performance information. Table 1 sets out the
documents and information relevant to the three spheres of government.
National Treasury 4
Chapter 2: Planning,
budgeting and reporting
The performance information reported in accountability documents enables Parliament, provincial legislatures,
municipal councils and the public to track government performance, and to hold it accountable.
Performance information also needs to be available to managers at each stage of the planning, budgeting and
reporting cycle so that they can adopt a results-based approach to managing service delivery. This approach
emphasises planning and managing with a focus on desired results, and managing inputs and activities to achieve
these results.
Performance information needs to be structured to demonstrate clearly how government uses available resources
to deliver on its mandate.
(a) Inputs: all the resources that contribute to the production and delivery of outputs. Inputs are "what
we use to do the work". They include finances, personnel, equipment and buildings.
(b) Activities: the processes or actions that use a range of inputs to produce the desired outputs and
ultimately outcomes. In essence, activities describe "what we do".
(c) Outputs: the final products, or goods and services produced for delivery. Outputs may be defined
as "what we produce or deliver".
(d) Outcomes: the medium-term results for specific beneficiaries that are the consequence of
achieving specific outputs. Outcomes should relate clearly to an institution's strategic goals and
objectives set out in its plans. Outcomes are "what we wish to achieve".
(e) Impacts: the results of achieving specific outcomes, such as reducing poverty and creating jobs.
When monitoring and assessing outcomes and impacts, it needs to be kept in mind that government interventions
can also have unintended consequences. These also need to be identified and monitored so that risks can be
managed and corrective action can be taken.
In managing for results, budgets are developed in relation to inputs, activities and outputs, while the aim is to
manage towards achieving the outcomes and impacts.
Figure 3 illustrates the relationship between these core performance information concepts.
National Treasury 6
Chapter 3: Key performance
i n f o r m a t i o n c o n c e p ts
Defining a good performance indicator requires careful analysis of what is to be measured. One needs to
have a thorough understanding of the nature of the input or output, the activities, the desired outcomes and
impacts, and all relevant definitions and standards used in the field. For this reason it is important to
involve subject experts and line managers in the process.
(a) Reliable: the indicator should be accurate enough for its intended use and respond to changes in
the level of performance.
(b) Well-defined: the indicator needs to have a clear, unambiguous definition so that data will be
collected consistently, and be easy to understand and use.
(c) Verifiable: it must be possible to validate the processes and systems that produce the indicator.
(d) Cost-effective: the usefulness of the indicator must justify the cost of collecting the data.
(e) Appropriate: the indicator must avoid unintended consequences and encourage service delivery
improvements, and not give managers incentives to carry out activities simply to meet a particular
target.
(f) Relevant: the indicator must relate logically and directly to an aspect of the institution's mandate,
and the realisation of strategic goals and objectives.
Institutions should include performance indicators related to the provision of goods and services. These describe
the interface between government and the public, and are useful for monitoring and improving performance as it
is relevant to the citizens of the country.
Figure 4 illustrates that performance indicators are relevant at all levels of the logic model. It also illustrates the
way in which economy, efficiency, effectiveness and equity are conceptualised.
Figure 4: Indicators of economy, efficiency and effectiveness, equity
Typical direct indicators include, cost or price, distribution, quantity, quality, dates and time frames, adequacy and
accessibility.
• Cost or Price indicators are both important in determining the economy and efficiency of service
delivery.
• Distribution indicators relate to the distribution of capacity to deliver services and are critical to
assessing equity across geographical areas, urban-rural divides or demographic categories. Such
information could be presented using geographic information systems.
• Quantity indicators relate to the number of inputs, activities or outputs. Quantity indicators should
generally be time-bound; e.g. the number of inputs available at a specific point in time, or the number of
outputs produced over a specific time period.
• Quality indicators reflect the quality of that which is being measured against predetermined standards.
Such standards should reflect the needs and expectations of affected parties while balancing economy
and effectiveness. Standards could include legislated standards and industry codes.
• Dates and time frame indicators reflect timeliness of service delivery. They include service frequency
measures, waiting times, response time, turnaround times, time frames for service delivery and timeliness
of service delivery.
• Adequacy indicators reflect the quantity of input or output relative to the need or demand - "Is enough
being done to address the problem?".
• Accessibility indicators reflect the extent to which the intended beneficiaries are able to access services
or outputs. Such indicators could include distances to service points, travelling time, waiting time,
affordability, language, accommodation of the physically challenged.
All government institutions are encouraged to pay particular attention to developing indicators that measure
economy, efficiency, effectiveness and equity using data collected through these and other direct indicators.
• Economy indicators: explore whether specific inputs are acquired at the lowest cost and at the right
time; and whether the method of producing the requisite outputs is economical. Economy indicators only
have meaning in a relative sense. To evaluate whether an institution is acting economically, its economy
indicators need to be compared to similar measures in other state institutions or in the private sector,
either in South Africa or abroad. Such indicators can also be compared over time, but then prices must be
adjusted for inflation.
• Efficiency indicators: explore how productively inputs are translated into outputs. An efficient operation
maximises the level of output for a given set of inputs, or it minimises the inputs required to produce a
given level of output. Efficiency indicators are usually measured by an input:output ratio or an
output:input ratio. These indicators also only have meaning in a relative sense. To evaluate whether an
institution is efficient, its efficiency indicators need to be compared to similar indicators elsewhere or
across time. An institution's efficiency can also be measured relative to predetermined efficiency targets.
• Effectiveness indicators: explore the extent to which the outputs of an institution achieve the desired
outcomes. An effectiveness indicator assumes a model of how inputs and outputs relate to the
achievement of an institution's strategic objectives and goals. Such a model also needs to account for
other factors that may affect the achievement of the outcome. Changes in effectiveness indicators are
only likely to take place over a period of years, so it is only necessary to evaluate the effectiveness of an
institution every three to five years; or an institution may decide to evaluate the effectiveness of its
different programmes on a rolling 3-5 year schedule.
National Treasury 8
Chapter 3: Key performance
i n f o r m a t i o n c o n c e p ts
• Equity indicators: explore whether services are being provided impartially, fairly and equitably. Equity
indicators reflect the extent to which an institution has achieved and been able to maintain an equitable
supply of comparable outputs across demographic groups, regions, urban and rural areas, and so on.
Often specific benefit-incidence studies will be needed to gather information on equity. The aim of such
studies would be to answer the question: "Who benefits from the outputs being delivered?" Usually
equity is measured against benchmark standards or on a comparative basis.
Institutions may also use the results of opinion surveys as indicators of their performance. Such indicators
should not replace the above two categories of indicators, but rather complement them. If an institution uses
such surveys, it is important that they be professionally designed.
Each institution needs to collect a wide range of performance information for management purposes,
however not all this information is relevant in accountability documents. The institution should specify in
its planning documents a set of performance targets it will report against in its accountability documents.
The set of indicators selected for accountability reporting ought to provide a holistic view of the
institution's performance.
In the case of concurrent functions, national departments need to identify a core set of indicators that need
to be reported by provincial and local governments to ensure comparability.
The baseline is the current level of performance that the institution aims to improve. The initial step in
setting performance targets is to identify the baseline, which in most instances is the level of performance
recorded in the year prior to the planning period. So, in the case of annual plans, the baseline will shift
each year and the first year's performance will become the following year's baseline. Where a system for
managing performance is being set up, initial baseline information is often not available.
This should not be an obstacle - one needs to start measuring results in order to establish a baseline.
Performance targets express a specific level of performance that the institution, programme or individual
is aiming to achieve within a given time period.
Performance standards standards express the minimum acceptable level of performance, or the level of
performance that is generally expected. These should be informed by legislative requirements,
departmental policies and service-level agreements. They can also be benchmarked against performance
levels in other institutions, or according to accepted best practices.
The decision to express the desired level of performance in terms of a target or a standard depends on the
nature of the performance indicators. Often standards and targets are complementary. For example, the
standard for processing pension applications is 21 working days, and a complementary target may be to
process 90 per cent of applications within this time.
An institution should use standards and targets throughout the organisation, as part of its internal
management plans and individual performance management system.
A useful set of criteria for selecting performance targets is the "SMART" criteria:
• Specific: the nature and the required level of performance can be clearly identified
• Measurable: the required performance can be measured
• Achievable: the target is realistic given existing capacity
• Relevant: the required performance is linked to the achievement of a goal
• Time-bound: the time period or deadline for delivery is specified.
National Treasury 10
Chapter 4
Even the best performance indicator information is of limited value if it is not used to identify service delivery and
performance gaps, to set targets and to work towards better results. Determining a set of appropriate indicators
depends on the nature of the institution's mandate.
Developing suitable performance indicators is a complex task. Six key steps may be identified in this approach:
Well-defined strategic goals and objectives provide a better basis from which to develop suitable programmes and
projects, as well as appropriate indicators. Once an institution has decided on what is to be achieved, it then needs
to decide what it needs to deliver to do so.
This approach to planning is called the "logic model", and is a useful way to plan and order information. In
determining the logic model, risk and assumptions must be identified for each of the levels of the planning process.
Specifying appropriate outputs often involves extensive policy debates and careful analysis. The process of
defining appropriate outputs needs to take into consideration what is practical and the relative costs of different
courses of action. It is also important to assess the effectiveness of the chosen intervention.
Ideally, targets should be set with reference to previous and existing levels of achievement (i.e. current baselines),
and realistic forecasts of what is possible. Where targets are set in relation to service delivery standards it is
important to recognise current service standards and what is generally regarded as acceptable.
This means getting the right information in the right format to the right people at the right time. Institutions need
to find out what information the various users of performance information need, and develop formats and systems
to ensure their needs are met.
Measuring, monitoring and managing performance are integral to improving service delivery.
National Treasury 12
Chapter 5
Effective management of performance information requires a clear understanding of different responsibilities, and
the structures and systems involved in managing performance.
5.1 Responsibilities
(a) Executive authorities: Ministers, MECs and mayors are accountable to Parliament, provincial
legislatures and municipal councils, and should provide these institutions with full and regular
reports concerning matters under their control. Ministers, MECs and mayors should in turn ensure
that the institutions under their control set up appropriate performance information systems so that
they are able to fulfil their accountability reporting responsibilities. They should also oversee such
systems to ensure that they are functioning optimally and comply with this Framework and other
related standards and guidelines.
(b) Accounting officers: The accounting officer or head official of an institution is accountable for
establishing and maintaining the systems to manage performance information. Their performance
agreements should reflect these responsibilities. They should be assisted by chief information
officers, and by ensuring there is appropriate capacity within the institution, as described in section
5.3 below.
(c) Line managers and other officials: Line managers are accountable for establishing and maintaining
the performance information processes and systems within their areas of responsibility. Their
performance agreements must reflect these responsibilities.
A range of officials is responsible for capturing, collating and checking performance data related
to their activities. The integrity of the institution's overall performance information depends on
how conscientiously these officials fulfil these responsibilities. Consequently, their performance
agreements and assessments should deal explicitly with the quality of this aspect of their work.
• Processes to set performance standards and targets prior to the start of each service delivery
period
6. Processes to ensure that responsibility for managing performance information is included in the
individual performance agreements of line managers and other officials
7. An identified set of performance indicators for reporting for oversight purposes.
It must be emphasised that line managers remain responsible for establishing and running performance
information systems within their sections, and for using performance information to make decisions.
National Treasury 14
Chapter 6
Institutions need to develop policies and procedures to publish performance information to meet these different
needs.
These "full and regular reports" are essentially the various accountability documents - the publication and
tabling of performance information in Parliament, provincial legislatures and municipal councils, linked to
the planning, budgeting, implementation and end-year reporting processes. Reporting responsibility rests
with the ministers, MECs or mayors, along with their accounting officers.
In certain cases the Constitution or legislation requires that reports be produced and tabled in Parliament,
such as the South African Human Rights Commission report on progress with the implementation of the
Bill of Rights.
These reports and publications are essentially secondary, since most performance information published is
sourced from the institutions responsible for gathering the information. This has implications for who
should be held accountable for the accuracy of the information. Ideally, the accounting officer or head
official of the institution from which the information was obtained should sign off on the information.
To minimise the duplication of reporting responsibilities and requests for information, coordination among the
oversight institutions is important. The Government-wide Monitoring and Evaluation System provides a
mechanism for improved coordination. The general approach adopted by the National Treasury is primarily to use
information that institutions publish in their accountability documents.
National Treasury 16
Chapter 6: Publishing performance
information
Generally, the institution that gathers the information should be the institution that publishes the information. A
suggestion for promoting transparency and accountability is that the key institutions responsible for line functions
should produce a "Statistical Annual" that provides detailed information on the functioning of the sector. For
example, each national department could produce a compendium of statistics relating to their area of responsibility.
• All previous and current accountability reports - strategic plans, operational plans, budgets, quarterly
performance reports, mid-term reports and annual reports
• Detailed performance information it holds that may be useful for decision-making in the private
sector and civil society
• Data sets of performance information for research purposes.
The Presidency and Premiers' Offices are among the key secondary users of performance information. As
such, they will use performance information collected, collated and reported by other institutions within
government to provide an overall picture of local, provincial and national performance.
In addition, the National Treasury and provincial treasuries are responsible for:
• Monitoring the implementation of the Framework by all institutions within their respective spheres
• Providing training on the use of performance information
• Providing input into the processes to select and define performance indicators
• Using the information generated by other institutions to monitor, evaluate and report on economy,
efficiency, effectiveness and equity in the use of resources to deliver services.
The national departments responsible for concurrent functions also need to play a supporting role, helping
provincial departments to manage performance information, and providing systems training.
National Treasury 18
Chapter 7: Roles and Responsibilities
The national departments should also monitor the performance information produced by their provincial
counterparts and use it to evaluate the overall delivery of services within their sector. The National
Treasury proposes that a product of this monitoring should be a "Statistical Annual" on service delivery by
sector, as suggested in section 6.3 above.
The department is responsible for developing and implementing an integrated monitoring, reporting and
evaluation system for local government, and for supporting the successful implementation of the
Government-wide Monitoring and Evaluation System. The DPLG is also responsible for the development
and implementation of monitoring, reporting and evaluation of the performance of provincial departments
of local government and municipalities.
CONCLUSION
The National Treasury will work with government departments and other institutions to identify performance
indicators that may be used for budget decision-making and for tracking service delivery against targets.
The National Treasury will also develop a number of guides and training materials to support the implementation
of this Framework.
National Treasury 20
Annexure 1
GLOSSARY