1. The Strategic Impact Evaluation Fund (SIEF)
BRIEF

Evaluations - Education

October 6, 2016

SEARCH OUR EVALUATIONS

 

BASIC EDUCATION SERVICE DELIVERY

 

Education is a powerful driver of development and one of the strongest routes for reducing poverty and ensuring equal opportunities for girls. While the number of children attending school has surged in the last decade, about 60 million primary-age children, mostly in developing countries, remain out of school. Enrollment is not the only problem. Students attending school are not learning: More than one of every three primary school students in the world cannot read, write, or do basic mathematics. Education reforms have traditionally focused on increased spending to improve the quality of education. However, such policies have done little for student learning. As such, policy attention is focusing on accountability, information, and incentives for both public and private schools. SIEF-supported researchers are working with governments and non-governmental organizations around the world to measure the impact of innovative programs to strengthen the quality of education, providing policymakers with evidence they can use to improve the lives of their students and the well-being of their countries. Read more about the background to our impact evaluations on accountability and education in the Cluster Note. Here is the background information about our more recent call for proposals on early childhood development and on engaging non-state actors in education. 

 

PORTFOLIO OF IMPACT EVALUATIONS 

 

Brazil: Measuring the Effectiveness of Pay Bonuses for Teachers on Students’ Performance

  • Timeline: Completed (Funded under SIEF1)
  • Evaluation: At the end of 2007, the São Paulo State Secretary of Education launched a program aimed at improving the quality of its 5,000 primary and secondary schools and 250,000 teachers. The evaluation sought to measure the impact of a key part of the reform program -- paying teachers bonuses linked to performance as a way to raise student test scores. Evaluation report pending.

 

Brazil: Using Teacher Feedback Program to Improve Learning: A SIEF-supported Impact Evaluation in Brazil

  • Principal Investigators - Barbara Bruns, World Bank; Leandro Costa, World Bank
  • Timeline: Completed
  • Evaluation: In Brazil, policy makers in the state of Ceara are looking at how providing information to schools about best teaching practices, as well as offering peer learning opportunities, can help boost the performance of less effective teachers. Researchers will evaluate the various methods in effort to locate low-cost alternatives to traditional, center-based training. 

 

Burkina Faso: Can Cash Transfers Help Children Stay Healthy and Go to School?

  • Timeline: Completed (Funded under SIEF 1)
  • Evaluation: Researchers conducted a randomized experiment to evaluate the impact of the Nahouri Cash Transfers Pilot Project in Burkina Faso’s Nahouri province, where poor families received either conditional or unconditional grants to encourage them to enroll their children in school and take them for regular check-ups. The program ran from 2008-2010.  The evaluation results helped the Government of Burkina Faso, working with the World Bank, develop a new social safety net program for the country. For further infromation, see our Burkina Faso Impact page.

 

Chad: Paying Community Teachers: Impact of the Payer and Transfer Mechanism

  • Principal investigators: Harounan Kazianga, Oklahoma State University; Leigh Linden, University of Texas at Austin; Helene Cloutier, World Bank
  • Timeline: Ongoing 
  • Evaluation: In Chad, almost all children go to primary school, but students perform poorly on assessments and many leave school before finishing. Teaching quality tends to be low, and many primary school teachers are contract teachers, not civil servants, and their payments are low and often delayed. As part of efforts to strengthen the education system, the Government of Chad is instituting a new way to pay contract teachers in rural parts of the country. The evaluation will measure the impact of different payment methods -- such as electronic payments through mobile phones -- on teacher behavior and student learning. 

 

Ghana: Supervision and Incentives for Increased Learning: The TCAI High Performance Program

  • Researchers: Anne Fitzpatrick, University of Massachusetts Boston; Adrienne Lucas, University of Delaware; Sabrin Beg, University of Delaware; Renaud Comba, Innovations for Poverty Action; Bridget Konadu Gyamfi; Innovations for Poverty Action; Joyce Jumpah, Innovations for Poverty Action; Deborah Newitter Mikesell, World Bank
  • Timeline: Ongoing
  • Evalution: Ghana has made major strides in education in recent years, registering a completion rate of nearly 100 percent for primary school and 81 percent for junior high in 2010-11. Despite large investments in basic education, however, learning outcomes have proven far from satisfactory. Fewer than one-third of primary school children have tested proficient in English or mathematics in recent years, and one-fifth of third-graders were unable to read a single word in English in a 2009 assessment. As part of a campaign to improve the quality of education, the Government is seeking to increase the amount of time primary school teachers spend using targeted instruction, which groups students by ability rather than grade. The evaluation will examine how increased training and supervision in this area impacts teacher performance and student success.

 

Guinea: Performance-Based Incentives for Teachers in Guinea

  • Researchers: Marie-Helene Cloutier, World Bank
  • Timeline: Completed
  • Evaluation: As in many developing countries, educating the poorest students in Guinea remains a challenge. Often, providing additional funding to schools is not enough to improve learning. The Government of Guinea wanted to improve student learning by enhancing teacher performance in primary schools through an incentive pilot scheme that included a financial reward, social recognition, and teacher training. Researchers evaluated the impact of these incentives on student achievement.

 

Haiti: Education Service Delivery Support

  • Principal Investigators: Moussa Blimpo, University of Oklahoma; Melissa Adelman, World Bank; David Evans, World Bank; Noah Yarrow, World Bank
  • Timeline: Completed
  • Evaluation: In Haiti, policy makers are looking for ways to increase teacher accountability in the education sector. Researchers worked with the Ministry of Education on a program that used cell phone cameras and wireless internet connections to verify teacher attendance. The evaluation helped policymakers understand some of the challenges of using technology in education.  

 

India: How to Make Performance Pay Work

  • Timeline: Completed (Funded under SIEF1)
  • Evaluation: Researchers set out to study what works better at improving student test scores—paying teachers bonuses based on results or giving schools cash grants for supplies or cash grants to hire one extra teacher. The test site was Andhra Pradesh, the fifth most populous state in India with more than 80 million people, of whom 73 percent live in rural areas. More than 80 percent of children in the rural parts of the state attend government-run schools.

 

India: Non-Financial Extrinsic and Intrinsic Teacher Motivation in Government and Private Schools in India

  • Principal investigators: Sangeeta Goyal, World Bank; Andrew Fraker, IDinsight; Neil Buddy Shah, IDinsight; Ronald Abraham, IDinsight; Deeptha Umapathy, IDInsight; Sangeeta Dey, World Bank; Lant Pritchett, Harvard University
  • Timeline:  2013 - 2017
  • Evaluation: In India, the rise in enrollment has been accompanied by a steady drop in student achievement:  the proportion of third-graders who know how to do subtraction, for example, decreased from 45 percent in 2006 to 28 percent in 2011. One reason is that the quality of teachers is low, with teachers themselves scoring poorly on math and language tests. To strengthen learning outcomes, NGOs seeking to boost education outcomes are experimenting with non-financial incentives to better motivate teachers using extrinsic motivation such as public recognition, as well as intrinsic motivations such as monthly meetings to help foster a sense of community and responsibility. The evaluation will gauge the cost-effectiveness of these strategies and their impact on teacher and student performance in both government and affordable private schools.

 

Kenya: Can Education Be Standardized?

  • Principal investigators: Guthrie Gray-Lobe, University of Chicago, Anthony Keats, Wesleyan University; Michael Kremer, University of Chicago; Isaac Mbiti, University of Virginia; Owen Ozier, Williams College
  • Timeline: 2016-2022
  • Evaluation: The last several decades have seen large increases in the number of students attending and completing primary school in Sub-Saharan Africa, but learning poverty remains high. Taking advantage of a lottery to give more than 10,000 scholarships to schools operated by the for-profit Bridge International Academies, researchers test the impacts of an approach to standardize and codify multiple components of the education system: pedagogy, school construction, and management.

 

Liberia: Can a Special Reading Program Help Liberia’s Children Learn?

  • Timeline: Completed (Funded under SIEF1)
  • Evaluation: After 14 years of civil war in Liberia, the education system was broken. There were few textbooks and other learning materials, and an estimated 62 percent of teachers had no formal training. Basic reading and math competencies among students were significantly lower than the rest of the region. The evaluation of Liberia’s Teacher Training and Early Grade Reading program sought to gauge the effectiveness of disseminating information about students’ reading ability to teachers and of giving teachers additional training as a way to improve students’ reading test scores.
  • Information for Accountability: Impact Evaluation of EGRA and teacher training (Evaluation Report)

 

Mexico: Empowering Parents to Improve Education

  • Timeline: Completed (Funded under SIEF1)
  • Evaluation: Improving school performance, especially in poor communities, remains a challenge facing most countries. The evaluation of Mexico’s Parental Empowerment Program sought to assess the effectiveness of giving more resources to parents’ associations in rural areas as a way to increase parental participation, boost school attendance and improve academic achievement.
  • Empowering Parents to Improve Education: Evidence from Rural Mexico (Evaluation Report)

 

Mexico: Impact Evaluation of a Low Cost Private School Model

  • Principal investigator: Lucrecia Santibanez, RAND Corporation
  • Timeline: Completed
  • Evaluation: Many public schools in Mexico are failing to educate students, particularly schools in poorer areas. Recent tests show that most 15-year-olds do not have basic competency in math, and almost 20 percent do not have basic reading skills. Reforms to improve education in Mexico move slowly, and parents rarely have a voice. This researchers evaluated the impact of Christel House de Mexico, a low-cost, rigorous private school for poor children that also works to ensure parental commitment. This study addressed the knowledge gap in the literature on the impact of private provision of high-quality, affordable schooling for the poor in developing country contexts. 

 

Mexico: Increasing Education Accountability through Community-Based Pedagogical Assistants

  • Principal Investigators:  Ciro Avitabile, World Bank; David Evans, World Bank; Peter Holland, World Bank
  • Timeline: Completed
  • Evaluation: In Mexico, where there is a large discrepancy in the quality of education between urban and rural schools, policy makers are looking for ways to decentralize decision making and encourage parents in indigenous communities to get more involved in their children’s schooling. As part of this effort, researchers are evaluating the effects of a program in Chiapas, one of the poorest states in Mexico,  that sends mobile pedagogical assistants to underperforming remote primary schools to give parents information about the schools and provide classroom support to teachers. The assistants, who are recent university graduates, review school performance, distribute education information throughout the community, and tutor struggling students. Researchers will evaluate the effects of the program on student learning. 

 

Morocco: Can Cash Transfers Help a Country Reach Universal Primary School Education?

  • Timeline: Completed (funded under SIEF 1)
  • Evaluation: Researchers worked with the Government of Morocco to measure the impact of a pilot cash transfer program to try to improve the primary school completion rate for poor children. The evaluation tested the effect of a small cash transfer – about five percent of a household’s annual expenditure – on schooling of children between the ages of six and 15. The pilot program was implemented over two school years, from 2008-2010. The transfer was divided into conditional and unconditional transfers (which were labeled as an educational support program), and then the study further divided transfers between mothers and fathers, to test whether it made a difference who received the money. The results showed that regardless of how the money was given and to whom, the cash transfers had a positive impact on children’s enrollment and attendance. The government has since continued the Tayssir program. For more information, visit our Morocco Impact page.  

 

Mozambique: Randomized Impact Evaluation of Various Early Literacy Interventions in Mozambique

  • Principal investigators: Marie-Helene Cloutier, World Bank; Sophie Naudeau, World Bank
  • Timeline: Completed
  • Evaluation: Studies have shown that teacher training combined with accountability strategies can result in improved student learning outcomes in early grades. More research is needed to disentangle the effect of the public information and training. The Government of Mozambique has implemented a pilot program to raise student achievement through teacher training and by providing families with information about reading test results. Researchers evaluated the effectiveness of these strategies on educational outcomes.

 

Nepal: The Impact of Decentralizing School Management

 

Nigeria: Understanding the Dynamics of Information for Accountability

  • Researchers: Olatunde Adekola, World Bank; Marie-Helene Cloutier, World Bank; Robert Garlick, Duke University; M. Abul Azad Alam, World Bank
  • Timeline: Cancelled 
  • Evaluation: Nigeria has launched key reforms in education, including a program making basic education compulsory. Enrollment rates, especially among the poorest families, are still insufficient in basic and secondary education, and the quality of schooling remains low.  In 2006, as part of a new round of reforms, the government unveiled a ten-year plan to improve access, equity and quality in education. One initiative aimed at high schools will provide parents with more information about the academic performance of their children’s schools using a combination of school meetings, flyers, text messages and phone calls.  Researchers will evaluate the impacts of this initiative, as well as the frequency of information delivery, on student performance and parent, teacher and school behavior. 

 

Pakistan: Community Engagement for School Committees in Pakistan

  • Principal investigators: Salman Asim, World Bank; Margo Hoftijizer, World Bank; Umbreen Arif, World Bank
  • Timeline: Completed
  • Evaluation: In Sindh province, only 61 percent of all children between the ages of 6-10 years are enrolled in school at the primary level. While on paper the province has a large number of schools, in practice many were found closed on an unannounced visit to school.  School management committees were designed to empower communities to advocate on behalf of their children’s education and bring about improvements in community’s schools. A pilot was designed to test the impact of steps to improve the functioning of the school committees by invigorating them through meetings and elections, and using text messaging platforms to help families and school officials receive and share information.

 

Pakistan: Helping Private Schools for Low-income Families Improve by Offering Loans

  • Principal investigators: Tahir Andrabi, Pomona College; Asim Khwaja, Harvard University; Jishnu Das, World Bank
  • Timeline: Completed
  • Evaluation: The World Bank team that’s been working in the education sector in Pakistan’s Punjab Province for more than a decade sought to determine whether making loans available to private schools that serve low-income students would lead schools to make investments that improve student learning. The study first tested the idea by offering schools a grant that was funded through a local institute.

 

Rwanda: Selection and Motivational Impacts of Performance Contracts for Rwandan Primary School Teachers

  • Principal Investigators: Owen Ozier, World Bank; Clare Leaver, University of Oxford; Pieter Serneels, University of East Anglia; Andrew Zeitlin, Georgetown University
  • Timeline: Ongoing
  • Evaluation: Pay-for-performance programs have become an increasingly popular way to boost teacher incetives and improve student learning. But little is known about the effects of these programs on the overall composition of a country’s teacher corps, especially in low income countries. Does the salary boost create incentives for more skilled and passionate young professionals to become teachers? Or does it have the opposite effect, in that it minimizes intrinsic motivation and instead attracts people interested only in a more attractive salary? Researchers are working with the Ministry of education to measure the impact of pay-for-performance in order to help recruit and keep better teachers. 

 

Senegal: Using Koranic Schools to Improve Education for Boys and Girls

  • Principal investigator: Jean Paul Pétraud, IMPAQ
  • Timeline: 2015-2019
  • Evaluation: Children in religious-based Koranic schools in Senegal, known as Daaras, don’t receive the basic education curriculum that is used in the formal, state sponsored education system. The Government of Senegal wants to boost academic achievement in Koranic schools, and is piloting a program to improve teaching through financial incentives, contributing to the cost of teaching materials and school infrastructure, and training and paying French-language teachers. The evaluation will measure the impact of this program on children’s learning, providing the government and donors with evidence they can use for future decision making.

 

Tanzania: Testing Information-for-Accountability and Teacher Incentive Interventions for Improving Education Service Delivery

  • Principal investigators: Shwetlena Sabarwal, World Bank; Deon Filmer, World Bank; James Habyarimana, Georgetown University
  • Timeline: Completed
  • Evaluation: In Tanzania, student learning has been hampered by high rates of teacher absenteeism. The Government of Tanzania sought to address these problems through non-financial performance-based incentives for teachers and improved information to community members and families on student and school performance. Researchers evaluated the effectiveness of these approaches.

Uganda: Measuring Approaches for Improving Teacher Performance

  • Principal investigators: Shwetlena Sabarwal, World Bank, James Habyarimana, Georgetown University, Felipe Barrera, Harvard University
  • Timeline: Completed
  • Evaluation: In 2007, Uganda became the first country in sub-Saharan Africa to implement a universal secondary education program, and since then, enrollment has jumped by 400 percent. The Ministry of Education and Sports has explored ways to improve teaching in both public and private schools by providing teachers with feedback, practical tips and non-financial rewards. The evaluation measured how these actions impact teacher and student performance.
RELATED
Welcome