BRIEF November 1, 2018

Nimble Evaluations

Armenia: How can we increase screening for non-communicable diseases?

Researchers: Damien de Walque, Adanna Chukwuma

Evaluation:  In Armenia, despite repeated efforts to encourage providers to screen for diabetes and hypertension and to encourage adults to go for testing, screening rates are still low for adults between the ages of 35 and 68. Using administrative data from health clinics and payment data, this evaluation will measure the impact of different approaches to increase take-up of screening, including a personal invitation sent in the mail, one-time cash payments, or lottery tickets for a prize.

 

Bangladesh: Can we improve learning outcomes by increasing the autonomy of bureaucrats?

Researchers: Mushfiq MobarakJaya WenMohammed Ashraful HaquePaula López-Pena

Evaluation: In many low-income countries, learning outcomes remain low despite increases in government spending on education. Financial incentives for students, teachers, and schools are often considered potential solutions for improving learning within classrooms, but government officials also play a key role in setting education policies.  One question is how this group can be encouraged to improve educational quality more effectively. Can increasing the decision-making autonomy of government officials who are responsible for schools lead to better learning outcomes? Can providing them with information on evidence-based approaches from other countries be helpful? This evaluation varies the autonomy of officials by rewarding them either for learning outcomes in general or rewarding them for taking specific actions that are thought to promote learning. It also will provide officials with information on well-tested educational programs from the scientific literature and provide them with incentives for introducing similar approaches. The final outcome of interest will be student scores on an annual national exam. 

 

Burundi: Expanding contraceptives use when women are shunned for using them

Researchers: Arndt Reichert

Evaluation:  The Government of Burundi wants to reduce the fertility rate by 2025 to three children per woman, from the current rate of 5.5 children per woman. Because there is a strong opposition to birth control from faith leaders, and because women who use birth control risk social isolation as a result, the government wants to introduce self-injectable contraceptives. But getting community health clinics ready to prescribe self-injectables and ensuring privacy for women will be challenging. Using data from Burundi’s health monitoring and information system, this evaluation will test different approaches for increasing adoption of these contraceptives, including financial incentives for community health volunteers and authorization for volunteers to provide the service in patients’ homes and instructions for patients to implement the necessary procedure themselves.

 

Cameroon: Increasing adoption of long-acting reversible contraceptives

Researchers: Susan Athey, Sarah Baird, Julian Jamison, Craig McIntosh, Berk Ozler, Dohbit Sama

Evaluation:  In sub-Saharan Africa, about one in four teenaged girls has been pregnant or had a baby. In Cameroon, 30 percent of teenaged girls who give birth say the pregnancy was unwanted at the time, but only a minority of sexually active unmarried women use any form of modern contraception. Less than one percent use a long-acting reversible contraceptive like an inter-uterine device or implant. A number of obstacles could be keeping adoption rates of long-acting reversible contraceptives low, such as lack of knowledge and bias against these among health care providers, sub-optimal subsidies to providers, and low ability or willingness to pay among adolescents. Using data from tablets used by nurses during family planning counseling sessions with patients, this evaluation will test the impacts of different approaches to increase adoption of long-acting reversible contraceptives, including training for nurses on modern contraceptive methods, a tablet-based decision-support app that takes patients’ preferences and circumstances into account, and the provision of free family planning services to adolescents. In a companion study, the research team will use machine learning techniques to tailor app recommendations to patients’ circumstances.

 

Dominican Republic: Nudging students to do better

Researchers: Melissa Adelman, Juan Baron

Evaluation: The Dominican Republic ranks among the lowest in student learning assessments, including globally in PISA 2015 and regionally in TERCE 2016. In addition to a comprehensive reform, the Ministry of Education is interested in identifying low-cost, scalable approaches to increasing student effort in day-to-day learning. Using data from a nationwide 9th grade diagnostic learning assessment, this evaluation will test the impacts of behaviorally-informed interventions to improve student motivation and effort, including an external (non-cash) incentive; growth-mindset exercises in which students learn that putting in effort, rather than fixed and immutable talent, can lead to high scholastic achievement; and self-affirmation exercises, in which students reflect on what is important to them and what about themselves makes them feel proud.

 

Ecuador: Increasing entrepreneurship and STEM careers through an online course in high school

Researchers: David McKenzie, Igor Asanov, Diego d’ Andria, Mona Mensmann, Bruno Crepon, Guido Buenstorf, Tom Astebro

Evaluation:  Entrepreneurship in high-growth sectors and employment in sectors requiring training in science, technology, engineering, and math (STEM) offer an opportunity for high-paying careers, but most young people in low-income countries do not select these educational and career paths.  In high school, students may not be aware of these career paths, they may lack the appropriate skills, and they may lack role models. This evaluation will test the impact of offering a 12-week online course offered in high school that includes content on personal initiative and negotiation skills, basic scientific methods, and role model interviews with entrepreneurs and scientists. Using data from an online monitoring system designed in tandem with the course, the evaluation will also test strategies for increasing course adoption among teachers, such as face-to-face versus online training, as well has different approaches for structuring how the material is presented, such as the order of content or adaptive exercises.

 

Estonia: Getting doctors to adopt quality enhancement practices for high-risk patients

Researchers: Kevin Croke, Benjamin Daniels, Daniel Rogger

Evaluation:  Estonia faces an aging population with an increasing burden of non-communicable diseases, such as hypertension, diabetes, and depression. From a physician’s perspective, care for patients at risk for these diseases can be quite intensive, as they require both preventive care and extensive follow-up care after diagnosis, as well as coordination among primary care physicians and specialists. Financial incentives for primary care physicians have failed to spur them to take-up quality enhancement programs, as have policies that have changed the role of primary care physicians vis a vis specialists. Using claims data from the Estonian Health Insurance Fund, this evaluation tests the impacts of strategies to increase physicians’ adoption of a quality enhancement program for high-risk patients, including a week-long mentoring visit from a peer-doctor with a structured curriculum, weekly one-hour coaching sessions, and a combination of the mentoring and coaching sessions.

 

Ghana: Encouraging parents to speak to their babies

Researchers: Pascaline DupasSeema JayachandranMark Walsh

Evaluation:  Although research shows the importance of stimulating infants and young children by talking to them, not all caregivers know to do this. In Ghana, where infant-directed speech is not common, researchers took advantage of an ongoing panel survey to implement simple interventions like videos promoting the importance of infant-directed speech and the provision of calendars as a reminder and tool to help caregivers track how much they speak to their infants. Since these interventions, along with a combination of the two interventions, were randomly assigned to survey respondents, in later phone surveys and direct assessments of children, researchers will be able to test the impact of these different approaches on parental views on speaking to children, parents’ self-reported practices, and children’s cognitive scores.

 

India: How can we reduce medical billing errors?

Researchers: Sebastian Bauhoff

Evaluation:  Health systems increasingly rely on administrative payment systems to reimburse service providers for claims they submit to insurers or governments.  In these systems, billing errors can threaten policy objectives and the viability of public programs, in addition to causing patients to lose money.  Errors may be mistakes, or they may indicate waste, abuse, or fraud. The issue is of great importance in India, which is in the process of unrolling a universal insurance plan for hospital care to 500 million people. This evaluation, which will rely on administrative data collected through the insurance program and cross-referenced with medical records and patient information, will test the impacts of providing hospital management with private “report cards” on their billing errors and  guidance on reducing errors;  informing patients about claims filed in their name; and a combination of the two interventions. 

 

Kenya: Will letting volunteer teachers choose their locations improve performance and boost national unity?

Researchers: Michael KremerWilla Friedman, Guthrie Gray-Lobe

Evaluation: Greatness United is a Kenyan government program that recruits and trains university graduates to assist low-performing public primary schools to improve student learning. It is also intended to boost national cohesion by placing volunteers in year-long assignments outside of their home counties. However, G-United faces high levels of applicant dropout, losing almost half of its applicants each year, mainly at assignment, and around the time they are to be sent to their placement areas. Using administrative data collected by the G-United program, this evaluation will test whether incorporating applicants’ geographic preferences in their placement can increase retention and lead to better performance. It will also test whether an assignment rule that takes geographic preferences into account affects ethnic and national identity. 

 

Lebanon: Supporting teachers to provide differentiated instruction to host and refugee students

Researchers: J. Lawrence Aber, Carly Tubbs Dolan, Lindsay Brown, Ha Yeon Kim, Alonso Sanchez, Mohamed Yassine

Evaluation: Due to the crisis in Syria, the formal school population in Lebanon doubled in a span of four years. To effectively teach in classrooms with a high number of vulnerable children, teachers may need training and support on differentiated instruction, providing psychosocial support to students, and socio-emotional learning. This evaluation will benchmark the impact of a teacher support program focused on continuous coaching supplemented with observation and feedback and scripted strategy cards for teachers’ coaches against the status quo coaching support system.

 

Madagascar: Testing the limits of integrating nutrition and early stimulation

Researchers: Lia Fernald, Emanuela Galasso, Lisy Ratsifandrihamanana, Ann Weber

Evaluation:  Parenting programs designed to improve how caregivers engage with their children have had mixed impacts on children’s development and mixed success in sustaining impact. In an earlier SIEF-funded evaluation of a pilot program in Madagascar, an early stimulation program that was integrated with a nutrition program and implemented by community health workers failed to demonstrate any impact on child development. One reason may have been difficulties faced by community health workers, who had to balance existing health and nutrition duties with new tasks related to explaining to parents how to support early childhood development. Using administrative data, GPS data from tablets carried by community health workers, direct observations, and surveys on how they use their time, this evaluation will look at how community health workers allocate their time across different activities while testing the impact of a new home visit and group meetings model to help boost children’s development.

 

Malawi: Deterring medication theft through information

Researchers: Brigitte SeimClark GibsonRyan JablonskiMariana Carvalho Barbosa

Evaluation:  The theft of medicines costs governments and donors billions of dollars annually. In Malawi, the government loses about 30 percent of the drugs and medical supplies it purchases to theft.  Despite years of donor-funded third-party distribution systems, a survey in 2015 found that 35 percent of private clinics were selling donor-supplied anti-malarial drugs that had been donated for free use. This evaluation will test the impact of different informational interventions on reducing drug theft, including providing information to clinic officials about a government tracking program to monitor drug supplies in each clinic and the penalties associated with stolen drugs; training communities to obtain information and monitor and report on drug delivery dates, drug availability, and clinic responsibilities; and a combination intervention.

 

Mozambique: Can an appointment system in health clinics and pharmacies increase HIV treatment adherence?

Researchers: Margaret McConnell, Sandra Sequeira, Maria StreenlandAmanda de Albuquerque

Evaluation:  Mozambique is among the countries with the highest HIV prevalence in the world. While most individuals living with HIV have initiated antiretroviral therapy, a third of patients do not remain on active treatment after 12 months. Patients often wait more than four hours to be seen at a public health clinic, where they go to get prescriptions renewed, and these long waiting periods may be contributing to poor treatment adherence. Building on a successful pilot in an antenatal care setting, this evaluation will use both administrative and survey data to test the impact of introducing scheduled appointment times on waiting times in clinics and pharmacies and on delays in return visits by HIV patients.

 

Mozambique: Can group-based incentives increase sewerage connections and adoption of improved sanitation?

Researchers:  Paul Gertler, Joshua Gruber, Alexander Spevack

Evaluation:  Central water treatment and sanitation services are largely responsible for eliminating outbreaks of waterborne diseases in industrialized countries. In Mozambique, only one in three households has access to safe water, and only one in ten to sanitation. This evaluation, centered on an ongoing World Bank project to improve water and sanitation in urban areas, will use a mix of administrative and survey data to test what form of financial subsidies increases sewerage connections and the use of improved sanitation: a high subsidy for households, a low subsidy for households, or subsidies offered to groups of households living in the same block that are valid only if a certain number of households sign up.

 

Pakistan: How can we improve nutrition counselling sessions by community health workers?

Researchers: Agha Ali AkramAkib KhanAbu Parves Shonchoy

Evaluation:  In Pakistan, almost half of children under the age of five are stunted from chronic malnutrition.  Stunting can have a negative impact on children’s cognitive development, health, and the years they spend in school, which together can reduce their chances for productive and healthy lives as adults. Nutrition counselling for parents through community health workers is a typical but not very effective way of addressing this. Using data collected by community health workers during their counselling visits, this evaluation will test different approaches for making counselling more effective, including growth charts that families can use to plot their child’s growth against healthy norms and small cash transfers that families are told – but not required – to use for purchasing healthy food for children.

 

Pakistan: Increasing adoption of water purification tablets

Researchers: Agha Ali Akram, Akib KhanJed FriedmanReshmaan Hussam  

Evaluation: Despite their potential to reduce diarrheal disease, use of cheap point-of-use decontamination technologies such as chlorine tablets is uncommon in many low-income countries, including in Pakistan, even when tablets are provided for free. This evaluation builds on a pilot in Pakistan that demonstrated a 56-percentage point increase in take-up of chlorine tablets when, using a simple chart, the health-workers facilitated household record-keeping of children’s diarrhea incidence to help them learn the benefits of chlorinated drinking water and provided information on diarrhea rates in other families that were not using chlorine tablets. Using data recorded by health workers when they visit households, the evaluation will test, at a larger scale, the impacts of helping households create these charts of their children’s diarrhea episodes, comparing their children’s episodes to those in peer households not using the chlorine tablets, and financial rewards for chlorine detected in households’ water.

 

Serbia: Can parenting programs increase their coverage and inclusiveness and still be effective?

Researchers: Pedro Carneiro, Ana Maria Munoz Boudet, Katia Herrera Sosa

Evaluation:  To prepare children for school, a stimulating home environment for learning and development is vital, since very young children spend most of their time at home. In Serbia, where fewer than 50 percent of young children are enrolled in preschool, the government is trying to expand preschool enrollment and at the same time promote stimulation at home by both mothers and fathers to ensure that poor children and those living in areas with low preschool access are better prepared to learn when they enter school. Parents will be offered a group program that will provide information on child development and activities to do at home; there will also be a communications campaign aimed at parents, particularly fathers, on how to promote young children’s development, and on the services available, such as preschools, and why children benefit from these early education programs. Using a combination of data sources, including nationwide household survey data collected by UNICEF, administrative records, and parents’ self-reports, this evaluation will test the relative effectiveness of the face-to-face and the communications campaign, as well as the impact of targeting on fathers, a group shown to be typically less involved in child engagement activities than mothers.