Building a Better Data Revolution

Last month the World Bank hosted the Survey Solutions Workshop, a four-day event to provide development practitioners with the tools and techniques to design and implement world-class household surveys. At the opening session, Marcelo Giugale, the World Bank's Director of Economic Policy and Poverty Reduction Programs for Africa, laid out the need for the workshop: “We must learn about the lives of the poor, and that cannot be done without timely, relevant, high-quality household survey data.” New technologies such as mobile phones and tablets are helping deliver on this agenda by providing more timely updates on the progress towards ending extreme poverty and boosting shared prosperity. But data quality remains elusive.

"If no emphasis is put on survey management, potential gains in quality stemming from the use of mobile technologies for data collection could be illusory," argued Talip Kilic, a research economist at the World Bank and part of the Living Standards Measurement Study (LSMS) team. "The data collected with tablets or smartphones may seem 'cleaner', but its accuracy could be questionable unless there is a system in place to hold all survey staff accountable, whether managers at the headquarters or staff in the field."

Ensuring accountability was one of the reasons for the creation of Survey Solutions, a unique suite of tools for data collection and household survey management. Survey Solutions was developed through a collaboration of the LSMS and Computational Tools teams within the World Bank’s research department. Although the market offers a range of software options to conduct surveys on tablets, none have sufficiently addressed the management aspects of fieldwork. Edwin St Catherine, director of statistics at St Lucia's Central Statistical Office, confirmed this point: "The possibility of managing enumerators and supervisors...is critical for us, critical to data quality, and it is not available in any of the other software."

Three years ago the World Bank came to the same conclusion. The LSMS group commissioned an independent review of available computer-assisted personal interviewing (CAPI) software. The conclusion? An all-inclusive CAPI solution that supported the survey management process was needed to enhance effectiveness in serving the World Bank’s clients—primarily statistical agencies in developing countries—in conducting high-quality household surveys.

Three years later Survey Solutions is now offering a one-stop shop for data collection: a suite of software tools for questionnaire design, survey management, and data collection, coupled with publications, training materials, and technical assistance delivered by LSMS survey specialists. "Starting the Survey Solutions project was, for us, a logical step. For more than thirty years we have accumulated extensive expertise in household surveys which we apply to promote best practices in data collection around the world, including through technological innovations and tools like Survey Solutions," explained Gero Carletto, manager of LSMS.

Over the course of the four-day workshop, representatives of nearly twenty national statistical offices and development agencies learned how to make the best use of the system's capabilities. The workshop offered a step-by-step walk-through of the system: creating user accounts for each field team member, assigning different roles to different team members, and allocating sampled households to team members to then conduct interviews. In mock exercises, questionnaires were administered to respondents on a tablet, and participants monitored field work and generated real-time reports highlighting team performance—exactly as it should happen in the field.

Workshop participants appreciated the practical approach of the workshop. Luqmaan Omar, executive manager of data processing at Statistics South Africa, summed it up: "18 hours in the plane wasn't easy, but it was definitely worth it. We have all the tools available and we are ready to kick off with planning.” More than 97% of the participants would like to attend such an event again; as many would recommend the sessions to a colleague.

While some of the attending experts were in the midst of running their own surveys with the management system—Survey Solutions has so far been implemented in 23 surveys in 11 countries—others were busy planning. Speaking on behalf of the World Bank’s Africa and South Asia Regions, which supported the workshop, John Newman, a lead statistician in the World Bank, provided participants with guidance on ways in which the World Bank, as well as the African Development Bank, United Nations Economic Commission for Africa, and the PARIS21 consortium can support implementation of a modern survey management system. At the same time, Michael Lokshin, manager of the Computational Tools team, discussed forthcoming features of the system that can further improve the statistical offices' workflows and lead to greater accountability for the data produced. 

Most of the participants have requested similar training events be held in their countries and regions. This is particularly important for people like M. Emdadul Haque, director of the Bangladesh Poverty Database, whose team is going to use Survey Solutions to manage the creation of a 36 million-large household database. "We have greatly benefitted from the workshop," he said, but he pointed out that more training will be required to successfully bring such a project to the field. "Our technical staff would immensely benefit from a follow-up event," agreed Goodson Sinyenga, deputy director of Zambia's statistical agency. As a result, the Computational Tools team is busy planning further trainings in the regions.