Learn how the World Bank Group is helping countries with COVID-19 (coronavirus). Find Out

BRIEF

Impact Evaluation to Inform and Transform Investments in Transport & ICT

June 10, 2015

Image

This is note #17 in the Connections series. You can download the PDF version of this note through this link.

By Nancy Vandycke, Arianna Legovini, Aleksandra Liaplina and Vincenzo di Maro

Image
A new World Bank initiative, “Impact Evaluation (IE) — Connect for Impact,” aims to radically transform and better inform the way that transport and information and communications technology (ICT) projects are designed and implemented.

Although multilateral lending to this key strategic sector comprises 29 percent of all global assistance, only 0.4 percent of impact evaluations have had transport as a subject. This initiative aims to fill the gap and bring high quality and valuable feedback to projects — improving design, enabling mid-course corrections and informing ex-post evaluations.

For the first time, this initiative will offer a systematic sector approach to generating concrete evidence of what works, what does not, where, when and why. It will greatly increase the impact and value-add of investments in transport and ICT projects— which is especially important given global trends toward increasing urbanization, with 70 percent of the world’s population expected to live in cities by 2050.

Introduction

Given their strategic importance to development, why have transport and ICT been so under-represented in impact evaluation? As places that attract infrastructure may differ from those that do not, it has been difficult to estimate the causal impact of investments. As a result, the assessment of investment programs’ outcomes in the transport sector has traditionally relied on descriptive analysis, failing to establish causal impact on job locations, migration patterns, land density, trade and market efficiency. Furthermore, it has been equally difficult to develop the cross-sector data infrastructure needed to measure outcomes.

Yet, the need for impact evaluations in these sectors is even greater now than in the past. ICT has introduced substantial innovations into every walk of life, including transport. And transport systems themselves have grown increasingly multifaceted, with an increasing portion of multilateral lending addressing more complicated development challenges, such as high-density multi-modal development corridors or urban mobility.

What is Impact Evaluation?

Impact evaluation (IE) demonstrates a causal link between an intervention and its impacts by comparing a treatment group with a control group. IE tests different interventions to identify the most effective treatment, and understand the mechanisms through which an intervention works (or not). This facilitates continuous experimentation and feedback into design and implementation — a real benefit over traditional evaluation. Current efforts in the Transport and ICT sectors build on IE methods developed and refined since early work in health and education, and more recently in sectors such as private sector development, agriculture, energy, and environment.

Impact Evaluation Contributions

Impact evaluations have proven to be an extremely powerful for maximizing returns on investments. The systematic analysis and intellectual rigor involved in IE can also give donors and governments added confidence in their decisions about current and future interventions. The benefits are many. IE’s evidence-based approach can improve delivery, increase efficiency, and boost impact. It can also answer many questions regarding which transport and ICT interventions can increase productivity, economic growth, and job creation. In addition, it can help in the design and implementation of interventions to promote environmentally-sustainable solutions to transport challenges.

Recent research demonstrates that the “delivery of projects with impact evaluation is significantly timelier: common delays are avoided and the gap between planned and actual disbursements is reduced by half.” Greater efficiency means quicker results on the ground, particularly vital in fragile and conflict-affected countries, as well as potential financial savings.

IE results can factor directly into project design. Such feedback can help determine whether adjustments need to be made: such as scaling up a specific intervention that has been very effective, or discontinuing the interventions which have been less so. IE can give policymakers a deeper understanding of design challenges, and the continuous feedback loop can help to better inform project design throughout the project cycle.

IE and the Transport and ICT Agenda

IE evidence can help with a host of new and increasingly large and complex sector challenges. Rapid urbanization will require substantial investments in improved public transport, urban planning, and car ownership and use—as well as in technology to reduce the need to travel. Getting urban transport systems right will be critical to address growing mobility needs.

High-density corridors, such as highways and national roads, freight and passenger railway links, inland waterways and airport infrastructure, constitute the backbone of a country’s transport network. Optimizing the transport network by looking at multimodality and spatial planning can support more efficient supply chains, reduce on-road freight transport costs, and facilitate regional integration.

Transport and ICT are critical for addressing greenhouse gas (GHG) emissions and moving towards more sustainable development. Transport accounts for 23 percent of GHG emissions, but its contribution to GHG emissions is expected to grow exponentially due to rapid urbanization and economic growth in developing countries.

“IE – Connect for Impact”

The World Bank’s new impact evaluation program for transport and ICT will develop an analytical framework for prioritizing and filling knowledge gaps. It will build the necessary knowledge and data for transport and ICT investments in general, and for conducting IEs in particular. It will also include iterative experimentation during implementation to inform mid-course decisions, improving delivery and effectiveness. Finally, the initiative will help build capacity for evidence-based decision-making across development banks and client country institutions.

Working in partnership with all relevant stakeholders, the Bank’s program will develop a sector-specific analytical framework for IE. It will pilot methodologies tailored to the sector which can become models for implementation. Most importantly, the success of this initiative is expected to drive the use of IE to design and implement better interventions that will benefit all concerned.

For more information on this topic:

DIME: www.worldbank.org/dime

DIME Policy Outreach Briefs: bit.ly/1FJwxvG

Gertler, Paul J.; Martinez, Sebastian; Premand, Patrick; Rawlings, Laura B.; Vermeersch, Christel M. J.. 2011. Impact Evaluation in Practice. World Bank. © World Bank. openknowledge.worldbank.org/handle/10986/2550

Arianna Legovini., Vincenzo di Maro and Caio Piza. 2015. Impact Evaluation Helps Deliver Development Projects. Policy Research. Working Paper WPS7157, World Bank, Washington, DC.

Connections is a weekly series of knowledge notes from the World Bank Group’s Transport & Information and Communication Technology (ICT) Global Practice. Covering projects, experiences, and front-line developments, the series is produced by Nancy Vandycke and Shokraneh Minovi. The notes are available at https://www.worldbank.org/en/topic/transport/brief/connections.