How do we do an impact evaluation? What needs to be considered? What the best approach? We’ve gathered together materials that can be useful to beginners an experts alike. Feel free to suggest new resources for us to include.
SIEF sponsors quarterly workshops to train regional policymakers, researchers and development field staff in the nuts and bolts of impact evaluation. Workshops use materials from the Impact Evaluation in Practice handbook available in English, French, Spanish and Portuguese. Materials from the workshops are available on our workshops page.
The building blocks for a successful and productive life are needed early on, when children’s brains and bodies begin to grow. Delays are hard to reverse. A solid body of evidence shows that young children who receive appropriate nutrition, health care and emotional and cognitive stimulation are better prepared for school and learning. Investing in children returns big payouts both in the short term and later on. Children who get the right start in life, starting with good nutrition and health care during their mothers’ pregnancies, will become healthier and more productive adults and are will be prepared for raising their own healthy children. SIEF-supported researchers are working to find the best and most cost-effective ways to deliver a range of early childhood nutrition, health and development programs in low-income countries. Read more about early childhood development and impact evaluation in our Cluster Note.
Burkina Faso: Reducing poverty and malnutrition in Burkina Faso through integrated early childhood interventions
India: Early Childhood Development for the Poor: Evaluating the Impacts in India
India and Pakistan: SPRING: Linking Implementation Strength, Outcomes and Lessons Learned to Inform Scale Up
Mali: Impact and cost effectiveness of an integrated parenting, nutrition and malaria prevention package in Mali
Education is a powerful driver of development and one of the strongest routes for reducing poverty and ensuring equal opportunities for girls. While the number of children attending school has surged in the last decade, about 60 million primary-age children, mostly in developing countries, remain out of school. Enrollment is not the only problem. Students attending school are not learning: More than one of every three primary school students in the world cannot read, write, or do basic mathematics. Education reforms have traditionally focused on increased spending to improve the quality of education. However, such policies have done little for student learning. As such, policy attention is focusing on accountability, information, and incentives for both public and private schools. SIEF-supported researchers are working with governments and non-governmental organizations around the world to measure the impact of innovative programs to strengthen the quality of education, providing policymakers with evidence they can use to improve the lives of their students and the well-being of their countries. Read more about the background to our impact evaluations on accountability and education in the Cluster Note. Here is the background information about our more recent call for proposals on early childhood development and on engaging non-state actors in education.
In many developing countries, weak health systems and the prevalence of often-preventable diseases undermine efforts to reduce poverty and improve opportunities for people to build better lives for themselves and their children. SIEF-supported researchers are working with health ministries and non-governmental organizations to measure the effectiveness of new and existing initiatives to strengthen access to and quality of care, including better services for women and girls. The evidence generated will make it possible for development practitioners and policymakers around the world to learn what’s working and how they can apply these lessons for better health care and coverage for all. Read more about health and impact evaluation in our Cluster Note.
Kenya: Advancing the Science of Delivery: A Proposed Impact Evaluation of Inspection Regimes in Health Care and Their Impact on Patient Safety Standards and Quality of Care
Tanzania:“RESPECT Study for women at high risk of HIV infection in Dar-es-Salaam, Tanzania”
Improved water supply and sanitation are critical for health and development, especially of young children, by reducing the transmission of disease. Toilets and easily accessible water supply systems provide women and children with security and dignity. They also can help raise school attendance, reduce healthcare costs and ultimately, strengthen productivity. For too many, this is still out of reach: More than 2.5 billion people don’t have access to basic sanitation, 1.1 billion people still defecate in the open, and more than 780 million people do not have regular access to clean drinking water. Improving water and sanitation makes good business sense as well, because poor sanitation and its effects depresses a country’s economic potential. SIEF-supported researchers are evaluating what mix of informational campaigns, financial instruments and incentives, and provision of sanitation infrastructure can successfully boost sustained access and usage. The evidence collected will give governments powerful evidence for making policy decisions that will ultimately bring tangible improvements in people’s well-being. Read more about water and sanitation and impact evaluation in our Cluster Note.
Ethiopia: The impact of enhanced, demand-side sanitation and hygiene promotion on trachoma and sustained behavior change in Ethiopia
India: Technical Proposal for the Impact Evaluation of Second Kerala Rural Water Supply and Sanitation Project (Jalanidhi-II)
Nicaragua Sustainable Water and Sanitation Sector Project
A key objective of SIEF is to strengthen the ability of researchers and policymakers to use monitoring and evaluation to build evidence on the effectiveness of programs to improve people’s lives through better health, nutrition, education, water and sanitation and social protection. SIEF sponsors quarterly workshops to train regional policymakers, researchers and development field staff in the nuts and bolts of impact evaluation. The goal is to encourage and support increased use of impact evaluation and to create a community of practice among development experts and policymakers. Workshops are invitation only and geared to specific development organizations, researchers, NGOs and government officials.
Workshops use materials from the Impact Evaluation in Practice handbook available in English, French and Spanish.
Ending extreme poverty and building shared prosperity requires evidence to identify those programs and policies that will have a real impact. The World Bank’s Strategic Impact Evaluation Fund (SIEF) makes this happen by investing in impact evaluations of innovative human development programs in low-and middle income countries, and by working directly with policymakers and other key stakeholders to use the results and build better policies and programs that successfully improve people’s lives.
SIEF is a multi-donor trust fund created in 2012 with the support of the British government’s Department for International Development (DFID) and currently also receives support from the London-based Children’s Investment Fund Foundation (CIFF), which seeks catalytic change for children including promoting early childhood development and evidence-based solutions. SIEF focuses on four human development areas that are crucial to improving the lives of the world’s poorest and most vulnerable: Early Childhood Development and Nutrition, Basic Education, Health Systems and Service Delivery, and Water Supply, Sanitation, and Hygiene.
SIEF partners with leading impact evaluation researchers and those who develop and implement innovative programs — both within governments and within non-governmental and other organizations. In the process, we ensure active engagement with key stakeholders and we support teams to make evidence accessible and policy relevant.
What is impact evaluation?
Knowing what works, what doesn’t, and why, is essential for crafting effective human development programs. Impact evaluation is a tool for measuring a program’s effectiveness. If policymakers know which programs help kids do better in school, increase maternal and child health, or boost employment, for example, they can build on those successes to create more—and more effective—programs to help the world’s poor.
Impact evaluations provide this evidence by comparing a program’s outcomes with what would have happened without the program, often referred to as a “counterfactual.” Specifically, evaluations compare beneficiaries of the program being evaluated with a comparison group of people sharing the same characteristics such as poverty level and education, for example, but who didn’t receive the program. This is usually done by randomly assigning the program intervention to control and treatment groups before the program is launched, and then comparing differences in outcomes once the program is implemented.
By isolating the impact of the program, impact evaluations provide policymakers and practitioners with valuable information for deciding whether they want to scale-up a program, change it or even cancel it.