Data and Analytics creates tools that improve the quality of impact evaluation research for all. We take advantage of the concentration and scale of research at DIME to develop and test solutions, first to ensure high quality of data collection and research quality across our portfolio, and, second, to make public training and tools available to the larger community of development researchers who might not have the same capabilities. The software tools, research guidelines, and trainings the Analytics team develops are all publicly available, as detailed below.
The DIME wiki is a one-stop shop for resources on all phases of an impact evaluation: design, fieldwork, data, and analysis. Each article contains a summary of best practices and key resources for successful execution of a particular impact evaluation task. All DIME Wiki content is publicly accessible and editable, and the DIME Analytics group has been active in recruiting contributions from other leaders in the field.
DIME Analytics’ flagship software package, ietoolkit, is a suite of Stata that routinize common analytical tasks in impact evaluations. It can be installed through SSC, and the source code is available for public review and contribution on GitHub. Demand for ietoolkit is clear: the software package has been downloaded an average of 304 times per month over the three months after its launch on the Development Impact blog. In March 2018, it ranked 3rd in GitHub trending Stata repository list.
GitHub is a version control tool that allows users to share and collaborate on code. The World Bank Group GitHub has two major types of repositories: research reproducibility resources for individual projects (such as Water When it Counts) and generalized code repositories with tools that are useful across projects (such as Stata code). These repositories allow researchers to make their code and research outputs public for transparency and as a resource to the development community.
DIME Analytics’ flagship training is a week-long annual course, open to the public. This hands-on training is designed to improve the skills and knowledge of impact evaluation implementers, familiarizing them with critical issues in IE implementation, recurring challenges, and cutting-edge technologies. In-person training spots are open to the public, with priority given to applicants from developing country governments, research institutions and non-profits. Remote access to the training, through WebEx, is open to everyone. All presentations and training materials from the course (including lab exercises and solutions) are publicly available through Open Science Framework.
The best practices documented in the DIME Wiki are also the subject of training sessions offered during the year to ensure that all DIME staff are familiar with the tools, materials and methods that guarantee data quality and research transparency. Some of the sessions held recently include best practices in data management, data cleaning and Stata coding as well research transparency tools such as LaTeX and GitHub. All the materials used for these trainings are made publicly available through the DIME Wiki.
Survey and code review
To ensure research reproducibility, the Analytics team reviews analytical code from DIME projects. Before data collections, we review survey forms and field plans and offer templates for data quality monitoring. During data processing, we promote a peer review process, where staff exchange and review each other’s code. Pre-publication, DIME Analytics reviews the final outputs and code to ensure that all published results are replicable.
Advancing the frontier
Annual measurement conference
For the last four years, DIME and the Center for Effective Global Action (CEGA) have convened researchers, policy-makers, and technology companies to discuss advances in the monitoring and measurement of economic indicators. Past convenings have focused on topics like environment and climate, disaster response, and infrastructure. The 2018 conference focused on artificial intelligence (AI), applied to a broad array of challenges in global development—from predicting migration patterns and crop yields, to detecting corruption, estimating poverty, and learning about consumers at the base of the economic pyramid.
New technologies for data collection
We aim to advance the knowledge frontier on data collection by identifying and capitalizing on opportunities to test new technologies. DIME Analytics’ role is to provide information and resources to teams interested in non-traditional data sources, and as possible direct pilot the technologies to provide first-hand information on challenges, costs, and use cases. Currently, a pilot is ongoing to test the use of Unmanned Aerial Vehicles (UAVs) for agricultural data collection. We are testing whether remote sensing can be a cost-effective supplement to traditional survey-based data collection in Rwanda.