|
 Under-utilization of evaluation findings and recommendations continues to be of concern to managers and evaluators in both developing and developed countries. Most evaluators have faced situations where methodologically sound evaluations with important operational and policy recommendations were never used. In some cases the decision-making processes were so complex that the evaluator did not even know if the findings were used.
The Bank's Independent Evaluation Group organized a workshop for IEG and other Bank staff, on ways to ensure that evaluations are used intensively and are as influential as possible. The workshop was delivered by Michael Bamberger, a retired Bank staffer with considerable experience in conducting evaluations and in ensuring their utilization.
The workshop was videotaped and is organized around four themes:
- what we mean by evaluation "utilization" and how it can be measured
- reasons why many evaluations are underutilized
- examples of evaluations that were considered influential
- guidelines for improving evaluation utilization.
Following brief introductions on each of these themes, participants discussed their own experiences concerning challenges and promising approaches in promoting the utilization of evaluations both within and outside the Bank. At the end of the workshop the lessons learned were reviewed. The workshop was chaired by Klaus Tilmes, the manager of IEG's Knowledge Programs and Evaluation Capacity Development Group.
The video of the workshop lasts for about 2 hours and 25 minutes, and is presented in individual segments covering the following topics:
Introduction and overview (28:35 minutes)
Defining and measuring utilization (13:44 minutes)
Reasons why evaluations are not used (10:13 minutes)
Examples of influential evaluations (13:48 minutes)
Ways to strengthen evaluation utilization (13:10 minutes)
Questions and answers — part 1 (12:08 minutes)
Questions and answers — part 2 (21:52 minutes)
Questions and answers — part 3 (21:09 minutes)
Questions and answers — part 4 (9:36 minutes)
The powerpoint slides used in the workshop are available for downloading. You might wish to print out a copy of these slides while you watch the video.
The references flagged at this workshop include:
 Bamberger, M., Rugh, J & Mabry, L. 2006. RealWorld Evaluation: Working under Budget, Time, Data and Political Constraints. Thousand Oaks, CA Sage Publications.
Government Accountability Office 1992. The Evaluation Synthesis. GAO/PEMD-10.1.2. Washington D.C.
Hatry, H.P. 2006. Performance Measurement: Getting Results. Second edition. Washington D.C: The Urban Institute.
Miles, M.B & Huberman, A.M. 1994. Qualitative Data Analysis. Thousand Oaks, CA. Sage Publications
Organisation for Economic Cooperation and Development 2001. Evaluation Feedback for Effective Learning. Paris. OECD/DAC.
Independent Evaluation Group. 2004. Influential Evaluations: Evaluations that Improved Performance and Impacts of Development Programs. World Bank.
Independent Evaluation Group. 2005. Influential Evaluations: Detailed Case Studies. World Bank.
Patton, M.Q. 1997. Utilization-focused Evaluation. Third edition. Thousand Oaks, CA. Sage Publications.
Patton, M.Q. 2002. Qualitative Research and Evaluation Methods. Thousand Oaks, CA. Sage Publications.
Vaughan, R.J. & Buss, T.F. 1998. Communicating Social Science Research to Policymakers. Thousand Oaks, CA, Sage Publications
Weiss, C.H. 1977. "Introduction" pp. 1-22. Using Social Research in Public Policymaking. Edited by Carol Weiss. Lexington MA. D.C. Health
Wholey, J.E., Scanlon, J.W., Duffy, H.G., Fukumotu, J.S. & Vogt, L.M. 1970. Federal Evaluation Policy: Analyzing the Effects of Public Programs. Washington D.C. The Urban Institute.
View the Videos | Download the Presentation
See also: Influential Evaluations
|
