Evaluation: In Armenia, despite repeated efforts to encourage providers to screen for diabetes and hypertension and to encourage adults to go for testing, screening rates are still low for adults between the ages of 35 and 68. Using administrative data from health clinics and payment data, this evaluation will measure the impact of different approaches to increase take-up of screening, including a personal invitation sent in the mail, one-time cash payments, or lottery tickets for a prize.
Azerbaijan: What type of messages improve take-up of health insurance?
Evaluation: Although Azerbaijan’s public health spending significantly increased over the last 15 years due to the country’s high oil revenues, it’s still below spending in comparable countries and insufficient for the population’s needs. As a result, out-of-pocket health spending is very high. In 2017, the Government of Azerbaijan launched a mandatory health insurance pilot to improve financing of health care and provide people with the health care services they need—including primary health care services, outpatient services, inpatient care, emergency care, diagnostics, laboratory, and surgeries. The pilot’s success led the President of Azerbaijan to issue a decree in December 2018 to expand the mandatory health insurance nationwide. The decree emphasizes Azerbaijan’s commitment to provide world-class health care to its population and the importance of introducing a mandatory health insurance system to improve the existing financing mechanisms and improve the quality of medical services. The question is now how to convince people to enroll in the national health insurance scheme once it’s rolled out in 2020. The government will soon start sending informational materials to people about the new system and enrollment -- this evaluation will test the impact of different informational “nudges” on people’s willingness to sign up.
Evaluation: In many low-income countries, learning outcomes remain low despite increases in government spending on education. Financial incentives for students, teachers, and schools are often considered potential solutions for improving learning within classrooms, but government officials also play a key role in setting education policies. One question is how this group can be encouraged to improve educational quality more effectively. Can increasing the decision-making autonomy of government officials who are responsible for schools lead to better learning outcomes? Can providing them with information on evidence-based approaches from other countries be helpful? This evaluation varies the autonomy of officials by rewarding them either for learning outcomes in general or rewarding them for taking specific actions that are thought to promote learning. It also will provide officials with information on well-tested educational programs from the scientific literature and provide them with incentives for introducing similar approaches. The final outcome of interest will be student scores on an annual national exam.
Researchers: Arndt Reichert
Evaluation: The Government of Burundi wants to reduce the fertility rate by 2025 to three children per woman, from the current rate of 5.5 children per woman. Because there is a strong opposition to birth control from faith leaders, and because women who use birth control risk social isolation as a result, the government wants to introduce self-injectable contraceptives. But getting community health clinics ready to prescribe self-injectables and ensuring privacy for women will be challenging. Using data from Burundi’s health monitoring and information system, this evaluation will test different approaches for increasing adoption of these contraceptives, including financial incentives for community health volunteers and authorization for volunteers to provide the service in patients’ homes and instructions for patients to implement the necessary procedure themselves.
Evaluation: In sub-Saharan Africa, about one in four teenaged girls has been pregnant or had a baby. In Cameroon, 30 percent of teenaged girls who give birth say the pregnancy was unwanted at the time, but only a minority of sexually active unmarried women use any form of modern contraception. Less than one percent use a long-acting reversible contraceptive like an inter-uterine device or implant. A number of obstacles could be keeping adoption rates of long-acting reversible contraceptives low, such as lack of knowledge and bias against these among health care providers, sub-optimal subsidies to providers, and low ability or willingness to pay among adolescents. Using data from tablets used by nurses during family planning counseling sessions with patients, this evaluation will test the impacts of different approaches to increase adoption of long-acting reversible contraceptives, including training for nurses on modern contraceptive methods, a tablet-based decision-support app that takes patients’ preferences and circumstances into account, and the provision of free family planning services to adolescents. In a companion study, the research team will use machine learning techniques to tailor app recommendations to patients’ circumstances.
Evaluation: The Dominican Republic ranks among the lowest in student learning assessments, including globally in PISA 2015 and regionally in TERCE 2016. In addition to a comprehensive reform, the Ministry of Education is interested in identifying low-cost, scalable approaches to increasing student effort in day-to-day learning. Using data from a nationwide 9th grade diagnostic learning assessment, this evaluation will test the impacts of behaviorally-informed interventions to improve student motivation and effort, including an external (non-cash) incentive; growth-mindset exercises in which students learn that putting in effort, rather than fixed and immutable talent, can lead to high scholastic achievement; and self-affirmation exercises, in which students reflect on what is important to them and what about themselves makes them feel proud.
Evaluation: Some 24 percent of children under the age of five in Ecuador are stunted, according to World Bank figures, and reversing this in the early years is critical to building human capital so all children can thrive and succeed. The Government of Ecuador is seeking to change these statistics by improving child nutrition during the first 1000 days of life, a key development period. They are revamping an existing nationwide program that sends trained staff, called Educadoras, door-to-door to make monthly home visits to families with young children to discuss prenatal care, healthy nutrition, breastfeeding, and hygiene. Using administrative data, the evaluation will test the impact of using text messages to improve Educadoras’ training and knowledge and reduce their drop-out rate during the four-week online training course. In addition, the evaluation will test whether sending text messages to parents to reinforce healthy behaviors can make a difference in children’s development, such as going for regular check-ups and completing required vaccinations.
Evaluation: Entrepreneurship in high-growth sectors and employment in sectors requiring training in science, technology, engineering, and math (STEM) offer an opportunity for high-paying careers, but most young people in low-income countries do not select these educational and career paths. In high school, students may not be aware of these career paths, they may lack the appropriate skills, and they may lack role models. This evaluation will test the impact of offering a 12-week online course offered in high school that includes content on personal initiative and negotiation skills, basic scientific methods, and role model interviews with entrepreneurs and scientists. Using data from an online monitoring system designed in tandem with the course, the evaluation will also test strategies for increasing course adoption among teachers, such as face-to-face versus online training, as well has different approaches for structuring how the material is presented, such as the order of content or adaptive exercises.
Evaluation: Estonia faces an aging population with an increasing burden of non-communicable diseases, such as hypertension, diabetes, and depression. From a physician’s perspective, care for patients at risk for these diseases can be quite intensive, as they require both preventive care and extensive follow-up care after diagnosis, as well as coordination among primary care physicians and specialists. Financial incentives for primary care physicians have failed to spur them to take-up quality enhancement programs, as have policies that have changed the role of primary care physicians vis a vis specialists. Using claims data from the Estonian Health Insurance Fund, this evaluation tests the impacts of strategies to increase physicians’ adoption of a quality enhancement program for high-risk patients, including a week-long mentoring visit from a peer-doctor with a structured curriculum, weekly one-hour coaching sessions, and a combination of the mentoring and coaching sessions.
Evaluation: Although research shows the importance of stimulating infants and young children by talking to them, not all caregivers know to do this. In Ghana, where infant-directed speech is not common, researchers took advantage of an ongoing panel survey to implement simple interventions like videos promoting the importance of infant-directed speech and the provision of calendars as a reminder and tool to help caregivers track how much they speak to their infants. Since these interventions, along with a combination of the two interventions, were randomly assigned to survey respondents, in later phone surveys and direct assessments of children, researchers will be able to test the impact of these different approaches on parental views on speaking to children, parents’ self-reported practices, and children’s cognitive scores.
Evaluation: Improving "last mile" public-service delivery is a recurring challenge in developing countries. Researchers evaluated the impact of using phone calls to monitor delivery of a program that transferred nearly a billion dollars to farmers in the Indian state of Telangana. A randomly selected sample of officials were told that a representative sample of beneficiaries in their jurisdiction would be called to measure the quality of program implementation. This simple announcement led to a 1.5 percent increase in the number of farmers receiving their benefits, with a 3.3 percent increase among farmers in the bottom quartile of landholdings. The program was highly cost-effective, with a cost of 3.6 cents for each additional dollar delivered.
Researchers: Sebastian Bauhoff
Evaluation: Health systems increasingly rely on administrative payment systems to reimburse service providers for claims they submit to insurers or governments. In these systems, billing errors can threaten policy objectives and the viability of public programs, in addition to causing patients to lose money. Errors may be mistakes, or they may indicate waste, abuse, or fraud. The issue is of great importance in India, which is in the process of unrolling a universal insurance plan for hospital care to 500 million people. This evaluation, which will rely on administrative data collected through the insurance program and cross-referenced with medical records and patient information, will test the impacts of providing hospital management with private “report cards” on their billing errors and guidance on reducing errors; informing patients about claims filed in their name; and a combination of the two interventions.
Evaluation: Greatness United is a Kenyan government program that recruits and trains university graduates to assist low-performing public primary schools to improve student learning. It is also intended to boost national cohesion by placing volunteers in year-long assignments outside of their home counties. However, G-United faces high levels of applicant dropout, losing almost half of its applicants each year, mainly at assignment, and around the time they are to be sent to their placement areas. Using administrative data collected by the G-United program, this evaluation will test whether incorporating applicants’ geographic preferences in their placement can increase retention and lead to better performance. It will also test whether an assignment rule that takes geographic preferences into account affects ethnic and national identity.
Evaluation: Due to the crisis in Syria, the formal school population in Lebanon doubled in a span of four years. To effectively teach in classrooms with a high number of vulnerable children, teachers may need training and support on differentiated instruction, providing psychosocial support to students, and socio-emotional learning. This evaluation will benchmark the impact of a teacher support program focused on continuous coaching supplemented with observation and feedback and scripted strategy cards for teachers’ coaches against the status quo coaching support system.
Evaluation: Parenting programs designed to improve how caregivers engage with their children have had mixed impacts on children’s development and mixed success in sustaining impact. In an earlier SIEF-funded evaluation of a pilot program in Madagascar, an early stimulation program that was integrated with a nutrition program and implemented by community health workers failed to demonstrate any impact on child development. One reason may have been difficulties faced by community health workers, who had to balance existing health and nutrition duties with new tasks related to explaining to parents how to support early childhood development. Using administrative data, GPS data from tablets carried by community health workers, direct observations, and surveys on how they use their time, this evaluation will look at how community health workers allocate their time across different activities while testing the impact of a new home visit and group meetings model to help boost children’s development.
Evaluation: The theft of medicines costs governments and donors billions of dollars annually. In Malawi, the government loses about 30 percent of the drugs and medical supplies it purchases to theft. Despite years of donor-funded third-party distribution systems, a survey in 2015 found that 35 percent of private clinics were selling donor-supplied anti-malarial drugs that had been donated for free use. This evaluation will test the impact of different informational interventions on reducing drug theft, including providing information to clinic officials about a government tracking program to monitor drug supplies in each clinic and the penalties associated with stolen drugs; training communities to obtain information and monitor and report on drug delivery dates, drug availability, and clinic responsibilities; and a combination intervention.
Evaluation: Mozambique is among the countries with the highest HIV prevalence in the world. While most individuals living with HIV have initiated antiretroviral therapy, a third of patients do not remain on active treatment after 12 months. Patients often wait more than four hours to be seen at a public health clinic, where they go to get prescriptions renewed, and these long waiting periods may be contributing to poor treatment adherence. Building on a successful pilot in an antenatal care setting, this evaluation will use both administrative and survey data to test the impact of introducing scheduled appointment times on waiting times in clinics and pharmacies and on delays in return visits by HIV patients.
Evaluation: Central water treatment and sanitation services are largely responsible for eliminating outbreaks of waterborne diseases in industrialized countries. In Mozambique, only one in three households has access to safe water, and only one in ten to sanitation. This evaluation, centered on an ongoing World Bank project to improve water and sanitation in urban areas, will use a mix of administrative and survey data to test what form of financial subsidies increases sewerage connections and the use of improved sanitation: a high subsidy for households, a low subsidy for households, or subsidies offered to groups of households living in the same block that are valid only if a certain number of households sign up.
Evaluation: In Pakistan, almost half of children under the age of five are stunted from chronic malnutrition. Stunting can have a negative impact on children’s cognitive development, health, and the years they spend in school, which together can reduce their chances for productive and healthy lives as adults. Nutrition counselling for parents through community health workers is a typical but not very effective way of addressing this. Using data collected by community health workers during their counselling visits, this evaluation will test different approaches for making counselling more effective, including growth charts that families can use to plot their child’s growth against healthy norms and small cash transfers that families are told – but not required – to use for purchasing healthy food for children.
Evaluation: Despite their potential to reduce diarrheal disease, use of cheap point-of-use decontamination technologies such as chlorine tablets is uncommon in many low-income countries, including in Pakistan, even when tablets are provided for free. This evaluation builds on a pilot in Pakistan that demonstrated a 56-percentage point increase in take-up of chlorine tablets when, using a simple chart, the health-workers facilitated household record-keeping of children’s diarrhea incidence to help them learn the benefits of chlorinated drinking water and provided information on diarrhea rates in other families that were not using chlorine tablets. Using data recorded by health workers when they visit households, the evaluation will test, at a larger scale, the impacts of helping households create these charts of their children’s diarrhea episodes, comparing their children’s episodes to those in peer households not using the chlorine tablets, and financial rewards for chlorine detected in households’ water.
Evaluation: To prepare children for school, a stimulating home environment for learning and development is vital, since very young children spend most of their time at home. In Serbia, where fewer than 50 percent of young children are enrolled in preschool, the government is trying to expand preschool enrollment and at the same time promote stimulation at home by both mothers and fathers to ensure that poor children and those living in areas with low preschool access are better prepared to learn when they enter school. Parents will be offered a group program that will provide information on child development and activities to do at home; there will also be a communications campaign aimed at parents, particularly fathers, on how to promote young children’s development, and on the services available, such as preschools, and why children benefit from these early education programs. Using a combination of data sources, including nationwide household survey data collected by UNICEF, administrative records, and parents’ self-reports, this evaluation will test the relative effectiveness of the face-to-face and the communications campaign, as well as the impact of targeting on fathers, a group shown to be typically less involved in child engagement activities than mothers.