Learn how the World Bank Group is helping countries with COVID-19 (coronavirus). Find Out

FEATURE STORY January 31, 2020

A Leaner Way of Monitoring Projects

Image

A school in Mali. - Photo: Ray Witlin/World Bank


It can be deeply challenging to determine if World Bank-funded projects are being well-run, especially in dangerous — and needy — parts of the world. While staff monitor projects through tracking surveys and site visits, neither method is without shortcomings. Since 2014, a new monitoring system that is light, low-cost, and secure has been successful in Mali and five other countries at filling in the gaps.

CHALLENGE

The World Bank task team leader in Mali was having doubts. It was 2014 and Mali has been experiencing instability and conflict since the military coup of 2012 and the occupation of the northern regions by armed groups.  School attendance plummeted, and a World Bank-funded project supplied meals at schools as a way to bring students back.

But reports were trickling in that some children weren’t being fed. The team leader needed to know if the reports were true. And he needed to know why. Site visits from a monitoring team were not turning up much. Those are telegraphed well in advance. A large-scale survey would be too expensive and take far too long.

Those issues, moreover, are not specific to any one country or any one project. Supervision missions can provide information, but the evidence they gather is limited to the small number of often carefully chosen projects they visit. Large-scale surveys are data-intensive, time-consuming and expensive. They often deliver results with such delay that their findings are no longer valid.

Another way was needed.

APPROACH

Johannes Hoogeveen, lead economist in the Poverty and Equity Global Practice, came up with a common-sense solution. He would harness the power of cell phone technology and the know-how of locals to get specific questions answered from hard-to-reach project sites. By cutting down the number of questions asked to a bare minimum, focusing on a select set of issues, sending out surveyors alone or in pairs, and collecting data quickly, he was able to get to ground truth.

It took short visits to just 20 schools in and around the capital city of Bamako over two weeks to get answers. Some of the money promised by the government had never been sent to the schools. The money that had been sent was wrong because the formula the government was using had errors. And the money that did get there was delayed. 

Armed with that information, the team leader was able to reach out to the government of Mali. Counts of students were improved. Payment delays were cut from three to one month. The number of annual disbursements tripled. 

The method Hoogeveen employed in Mali, which he dubbed Iterative Beneficiary Monitoring (IBM), is not complex. Indeed, the whole idea behind it is to strip away the layers of systems that can add cost and intricacy to the business of evaluations without adding to a real base of knowledge of what is happening on the ground. For a fraction of the cost of traditional surveys and site visits, IBM sends local people, armed with several dozen questions on computer tablets, out to hard-to-reach project sites. Questioners go alone on public transport instead of in large groups in expensive four-wheel-drive vehicles. Whenever possible sending enumerators is avoided and interviews are held over the phone. The low cost of data collection makes it possible to go check back every few months and assess whether improvements are sustainable.

“Getting ground truth about these projects is important because without it, we can never fix the wrongs that inevitably happen, the problems that arise even in the best of circumstances,” Hoogeveen said. “Nothing is designed perfectly. I firmly believe that if something is implemented the way you thought it through behind your desk, then you’re doing something very wrong. If you want to create change, you probably need much more frequent feedback. You need that calibration until you get a working system.”


Image

Children in the Malian town of Bafoulabé. - Photo: Sarah Farhat/The World Bank


RESULTS

Since IBM was first introduced in Mali, it has been used to monitor 30 projects in six countries, including Chad and Niger. At a cost of less than $5,000 per project, the monitoring system has helped improve project effectiveness from fertilizer subsidies to medical care.

In Mali and Niger, for example, the government gave farmers vouchers, funded by the Bank, to acquire fertilizer. Employing IBM, Bank staff discovered the vouchers had been distributed too late in the year for them to be useful.  The Bank delayed approval of its budget support until the next agricultural season, when these problems were fixed. 

The monitoring also uncovered another problem. In project after project, cash, vouchers and other benefits were distributed almost exclusively to men, despite the best intentions of Bank management. It turns out targeted beneficiaries were heads of household which are usually men. 

Armed with that information, Bank staff were able to take steps to ensure benefits were distributed to both genders.

WORLD BANK GROUP CONTRIBUTION

The IBM data collection efforts were initially funded by the Mali program office and have been bolstered by an innovation grant from the Poverty and Equity Global Practice. A small team of economists designs the short questionnaires, then hires local consultants in each country to conduct interviews in person and by phone. 

MOVING FORWARD

IBM is not designed to replace other systems used to monitor the progress of World Bank projects. But it can be used to improve them.  A focus on IBM and other ways of monitoring projects that are leaner, less obtrusive and yield faster results is the future of development, Hoogeveen said. 

“We want this to become redundant,” Hoogeveen said. “We should push to improve our monitoring systems across the board.”



Api
Api