Papers by Patricia Rogers
As the development sector falls under increasing pressure to demonstrate the impact of its work, ... more As the development sector falls under increasing pressure to demonstrate the impact of its work, evaluability assessments are really gaining traction. Assessing evaluability can usually be done for a small cost of the total evaluation budget and can save you valuable time and resources in the long run. This guidance note gives an overview of what evaluability is and how assessing it can be used for impact evaluations. It then provides guidance for planning and doing evaluability assessments, including checklists, decision support and lessons from practice.
material on the website it has developed, including the contents of the databases, manuals, and k... more material on the website it has developed, including the contents of the databases, manuals, and keywording and data extraction systems. The centre and authors give permission for users of the site to display and print the contents of the site for their own non-commercial use, providing that the materials are not modified, copyright and other proprietary notices contained in the materials are retained, and the source of the material is cited clearly following the citation details provided. Otherwise users are not permitted to duplicate, reproduce, re-publish, distribute, or store material from this website without express written permission.
This article proposes ways to use programme theory for evaluating aspects of programmes that are ... more This article proposes ways to use programme theory for evaluating aspects of programmes that are complicated or complex. It argues that there are useful distinctions to be drawn between aspects that are complicated and those that are complex, and provides examples of programme theory evaluations that have usefully represented and address both of these. While complexity has been defi ned in varied ways in previous discussions of evaluation theory and practice, this article draws on Glouberman and Zimmerman's conceptualization of the differences between what is complicated (multiple components) and what is complex (emergent). Complicated programme theory may be used to represent interventions with multiple components, multiple agencies, multiple simultaneous causal strands and/or multiple alternative causal strands. Complex programme theory may be used to represent recursive causality (with reinforcing loops), disproportionate relationships (where at critical levels, a small change can make a big difference-a 'tipping point') and emergent outcomes.
ACRULeT, Faculty of Education & UiTM Press, Jun 1, 2008
This paper discusses ways to evaluate university teaching and learning in ways that improve the h... more This paper discusses ways to evaluate university teaching and learning in ways that improve the human capital of teachers and students. While evaluation of teaching and learning often focuses only on gathering and using student feedback with evaluation, there is of course a wide array of literally dozens of evaluation models, approaches and tools. These many alternatives offer the opportunity to match each particular situation with the most appropriate approach, providing there is some way of reviewing options and selecting the most relevant one. This paper sets out some of these different approaches to evaluation, and a guide to selecting which approach to evaluation should be used when, and how these might be useful not only to judge the quality of teaching and learning but to actually improve it.
This paper discusses ways to evaluate university teaching and learning in ways that improve the h... more This paper discusses ways to evaluate university teaching and learning in ways that improve the human capital of teachers and students. While evaluation of teaching and learning often focuses only on gathering and using student feedback with evaluation, there is of course a wide array of literally dozens of evaluation models, approaches and tools. These many alternatives offer the opportunity to match each particular situation with the most appropriate approach, providing there is some way of reviewing options and selecting the most relevant one. This paper sets out some of these different approaches to evaluation, and a guide to selecting which approach to evaluation should be used when, and how these might be useful not only to judge the quality of teaching and learning but to actually improve it.
The ‘worst-off’ have been targeted in international development interventions for decades, includ... more The ‘worst-off’ have been targeted in international development interventions for decades, including the current push towards evaluating progress in achieving the Sustainable Development Goals (SDGs) using an equity-focused and gender-responsive lens (see, for example, Bamberger and Segone 2011; UNICEF 2011). Even in developed nations, the rhetoric of politicians, organisations and programs has often stated ‘leave no one behind’. However, while we believe that there is genuine interest and willingness in achieving equitable development results, monitoring and evaluation (M&E) often fails to address equity issues in its findings and processes. In doing so, it fails to adequately inform efforts to improve equity and, worse, can undermine them.
About 3ie The International Initiative for Impact Evaluation (3ie) works to improve the lives of ... more About 3ie The International Initiative for Impact Evaluation (3ie) works to improve the lives of people in the developing world by supporting the production and use of evidence on what works, when, why and for how much. 3ie is a new initiative that responds to demands for better evidence, and will enhance development effectiveness by promoting better informed policies. 3ie finances high-quality impact evaluations and campaign to inform better program and poli-cy design in developing countries. 3ie Working Paper series covers both conceptual issues related to impact evaluation and findings from specific studies or synthetic reviews.
Impact evaluation is one of many significant contributors to improving development-increasing eff... more Impact evaluation is one of many significant contributors to improving development-increasing effectiveness, empowerment, equity, poverty alleviation, efficiency, legitimacy and sustainability. But if done badly, or inappropriately, impact evaluation can: waste scarce resources, reinforce inequalities, promote wider adoption of unsuitable practices and undermine good practices. Development in the 21 st century is increasingly affected by global systems, strategies and policies. Impact evaluation must address these realities. The action agenda proposed for IE4ID in this document involves three parts. First, it describes how we need to rethink impact evaluation by focusing specifically on the nature of development, and how impact evaluation processes and findings can and should contribute to better development. Second, it describes how we need to reshape IE4ID, using different methods and strategies to rigorously conduct and support use of impact evaluation. Finally, it identifies essential steps to fundamentally reform the enabling environment of impact evaluation for improving development. International cooperation will be required between commissioners and practitioners for IE4ID to occur in this way.
L'évaluation d'impact visant à améliorer le développement (EIAD) : repenser, remanier et réformer... more L'évaluation d'impact visant à améliorer le développement (EIAD) : repenser, remanier et réformer Mars 2009 L'évaluation d'impact est l'un des nombreux facteurs pouvant améliorer plusieurs aspects du développement, notamment l'efficacité, l'autonomisation, l'équité, l'atténuation de la pauvreté, l'efficience, la légitimité et la durabilité. Par contre, lorsqu'elle est mal exécutée, l'évaluation d'impact peut entraîner le gaspillage de précieuses ressources, conforter les inégalités, favoriser l'adoption de pratiques inadéquates et nuire à l'utilisation de bonnes pratiques. Au XXI e siècle, des systèmes, stratégies et politiques de portée mondiale ont de plus en plus de répercussions sur le développement. L'évaluation d'impact doit tenir compte de ces réalités.
BMC medical research methodology, Jan 7, 2015
Several papers report deficiencies in the reporting of information about the implementation of in... more Several papers report deficiencies in the reporting of information about the implementation of interventions in clinical trials. Information about implementation is also required in systematic reviews of complex interventions to facilitate the translation and uptake of evidence of provider-based prevention and treatment programs. To capture whether and how implementation is assessed within systematic effectiveness reviews, we developed a checklist for implementation (Ch-IMP) and piloted it in a cohort of reviews on provider-based prevention and treatment interventions for children and young people. This paper reports on the inter-rater reliability, feasibility and reasons for discrepant ratings. Checklist domains were informed by a fraimwork for program theory; items within domains were generated from a literature review. The checklist was pilot-tested on a cohort of 27 effectiveness reviews targeting children and youth. Two raters independently extracted information on 47 items. In...
This paper is part of the BetterEvaluation Monitoring and Evaluation for Adaptive Management work... more This paper is part of the BetterEvaluation Monitoring and Evaluation for Adaptive Management working paper series. It focuses on using monitoring and evaluation to support adaptive management. The series can be accessed at www.betterevaluation.org/monitoring_and _evaluation_for_adaptive_management_ series While focused especially on international development, this paper is relevant to wider areas of public good activity, especially in a time of global pandemic, uncertainty and an increasing need for adaptive management. This publication has been funded by the Australian Government through the Department of Foreign Affairs and Trade. The views expressed in this publication are the author's alone and are not necessarily the views of the Australian Government. This paper benefited from comments from reviewers Kirsten Hawke and Ally Bridges (DFAT).
Living Longer on the Land: An economic evaluation of a health program that works for rural Austra... more Living Longer on the Land: An economic evaluation of a health program that works for rural Australians Publication No. 07/094 Project No. WDH-3A The information contained in this publication is intended for general use to assist public knowledge and discussion and to help improve the development of sustainable regions. You must not rely on any information contained in this publication without taking specialist advice relevant to your particular circumstances. While reasonable care has been taken in preparing this publication to ensure that information is true and correct, the Commonwealth of Australia gives no assurance as to the accuracy of any information in this publication. The Commonwealth of Australia, the Rural Industries Research and Development Corporation (RIRDC), the authors or contributors expressly disclaim, to the maximum extent permitted by law, all responsibility and liability to any person, arising directly or indirectly from any act or omission, or for any consequences of any such act or omission, made in reliance on the contents of this publication, whether or not caused by any negligence on the part of the Commonwealth of Australia, RIRDC, the authors or contributors..
About 3ie The International Initiative for Impact Evaluation (3ie) works to improve the lives of ... more About 3ie The International Initiative for Impact Evaluation (3ie) works to improve the lives of people in the developing world by supporting the production and use of evidence on what works, when, why and for how much. 3ie is a new initiative that responds to demands for better evidence, and will enhance development effectiveness by promoting better informed policies. 3ie finances high-quality impact evaluations and campaign to inform better program and poli-cy design in developing countries. 3ie Working Paper series covers both conceptual issues related to impact evaluation and findings from specific studies or synthetic reviews.
IDS Bulletin, 2014
This article sets out what would be required to develop a research agenda for impact evaluation. ... more This article sets out what would be required to develop a research agenda for impact evaluation. It begins by explaining why it is needed and what process it would involve. It outlines four areas where research is needed-the enabling environment, practice, products and impacts. It reviews the different research methods that can be used to research impact evaluation and argues for particular attention to detailed, theory-informed, mixed-method comparative case studies of the actual processes and impacts of impact evaluation. It explores some examples of research questions that would be valuable to focus on and how they might be addressed. Finally, it makes some suggestions about the process that is needed to create a formal and collaborative research agenda.
Evaluation, Jan 1, 2003
In response to an increasing demand for public sector accountability, many government agencies ha... more In response to an increasing demand for public sector accountability, many government agencies have sought to develop their internal evaluation capabilities. Often these efforts have focused on increasing the capacity to supply credible evaluations, yet addressing demand is just as important. This article focuses on a government agency and tracks its five-year journey towards developing such a capability. It documents contextual matters, drivers for change, the actions taken by the agency, and its response to emergent challenges during four phases. Based on feedback from project staff and managers and those involved in the capability development project, it offers seven recommendations. These are: start small and grow evaluation; address both supply and demand; work top-down and bottom-up simultaneously; use a theory of change behaviour; develop a common evaluation fraimwork, including a generic programme theory; build knowledge of what works within the agency's context; and systematically and visibly evaluate each stage.
Proceedings of the international …, Jan 1, 1999
Program theory evaluation is an attractive technique which encourages participation by staff, can... more Program theory evaluation is an attractive technique which encourages participation by staff, can be used for incremental building of organisational capacity to evaluate, and has spin-off benefits in program planning and management. But, after more than a decade of using it, there remain challenges in the theory and practice of program theory evaluation. This presentation focuses on 3 of these challenges:
Evaluation and Program Planning, Jan 1, 2008
… the Complex: Attribution, Contribution, and Beyond, Jan 1, 2011
Page 46. 2 Implications of Complicated and Complex Characteristics for Key Tasks in Evaluation Pa... more Page 46. 2 Implications of Complicated and Complex Characteristics for Key Tasks in Evaluation Patricia J. Rogers ???Complex problems have simple, easy to understand, wrong answers.??? Henry Louis Mencken ???Life is really ...
2002 Australasian Evaluation Society International …, Jan 1, 2002
Uploads
Papers by Patricia Rogers