Challenge
The potential gains from bringing existing technologies into a country like Colombia are vast. Differences in technology adoption can account for up to 75 percent of the income gap across countries. Despite the potential benefits, a recent World Bank report, “The Innovation Paradox,” argues that developing economies are often unable to reap the potentially large gains from innovation and technological catch-up due to institutional failures and missing complementary factors.
The array of available instruments and the quality of policy implementation for entrepreneurship and innovation vary significantly across countries, and in low- and middle-income countries such instruments and policies tend to be highly fragmented. Very often, new instruments are introduced to enhance the quantity and quality of private sector innovation despite limited evidence for their impact. Often, responsibility for implementation of the policy mix is spread across several government agencies, resulting in problems of duplication, incomplete access, and information asymmetry across policy practitioners.
Approach
In 2015, Colombia’s policy practitioners sought advice from the World Bank Group to improve their policy mix of programs supporting science, technology, and innovation (STI). Together with counterparts from the National Department of Planning (Departamento Nacional de Planificación, DNP), the World Bank implemented a methodology to identify ways to improving the design, implementation, and effectiveness of Colombia’s STI programs.
The Public Expenditure Review (PER) for Science, Technology, and Innovation methodology, first envisioned in 2014 by the World Bank’s Trade and Competitiveness global practice, has evolved to include a broad set of public programs supporting private sector development, some of which target entrepreneurship and SMEs. Using policy instruments as the unit of analysis (in stark contrast to the traditional aggregate budget headings used for PER), the methodology proceeds in four stages:
- Mapping and evaluation of the quality of policy mix
- Functional and governance analysis
- Efficiency analysis
- Effectiveness analysis
In Colombia, the first two stages were piloted in collaboration with DNP in 2015–2016, with assessments of 129 STI programs managed by eight agencies. The policy practice analysis employed a public management scorecard that assessed performance across 27 dimensions of good practices in design, implementation, and governance.
In 2018, the first two stages were adapted for implementation at the subnational level, with the subject of analysis set as the portfolio of projects in the Science, Technology and Innovation Fund (Fondo de Ciencia, Tecnología e Innovación, FCTeI), which by constitutional mandate receives 10 percent of royalties from non-renewable resources. The implementation of these exercises led to significant changes in government agencies’ program designs and in policy strategy coordination across stakeholders. Through guided interviews and focal groups, a rich set of information was gathered from portfolio managers and project implementers. In the first stage, 166 projects in 12 project portfolios (corresponding to 12 departments) were mapped and analyzed according to 10 areas, among them, budget, partnership types, economic activity, intervention mechanisms, economic results, STI objectives, and beneficiaries. The portfolios were also evaluated for functional performance based on 25 variables in three dimensions (design, implementation, and governance).
The efficiency analysis, PER’s third phase, was later incorporated and implemented in 2017 at the national level, enabling policy practitioners to understand how well programs were employing their financial and human resources to deliver results.
Overall, the analytical framework enables policy makers to identify opportunities for improving delivery based on metrics in the following categories: (i) use of inputs; (ii) materialization of outputs; (iii) achievement of outcomes; and (iv) beneficiary satisfaction, which incorporates a user-centric perspective on the performance of programs’ selection, delivery, and results.
Results
· Policy decisions were better informed: Policy mapping enabled agencies to realize that goals and beneficiaries of some programs significantly overlapped, with little coordination occurring between them. Following the recommendation to simplify the project portfolio and provide further transparency, the Government of Colombia created a digital platform, termed the Innovation Portal, to provide detailed information about different instruments supporting STI and entrepreneurship and about overall policy strategy at the national level.
· The portfolio concept was adapted to total public investment planning: Based on the PER methodology applied at the national (2015) and subnational (2018) levels, DNP developed its Articulation for Competitiveness (Articulación para la competitividad, ArCo) methodology, which takes a portfolio approach to ex-ante (instead of ex-post) evaluation of instruments financed by the National General Budget. DNP expects that implementing this methodology will lead to measurable efficiency gains in the investment national budget. A pilot will be implemented in 2020. In addition, at the subnational level the portfolio concept was incorporated into the FCTeI project cycle to accomplish the following: (i) incorporate a portfolio vision for the definition of investment priorities in departmental STI councils (Codectis); (ii) define terms of reference for calls for projects; (iii) update the STI sector guide for departments; and (iv) adjust department-level decrees on STI strategic plans.
· STI policy at the subnational level was strengthened: Some recommendations of the subnational PER led to changes in STI policy strategies. For instance, the Article 165 of the National Development Plan Law enables the use of FCTeI resources toward credit lines for STI activities through second-tier banking and development of Catedra CTI (a STI curricula) by the STI ministry, which seeks to strengthen territorial capacities by training project managers in CTI.
· Programs were rationalized: Reducing the number of instruments allowed agencies to sharpen the focus of their programs, thus reducing administrative costs. For example, INNpulsa, an agency responsible for supporting private sector development, consolidated previously overlapping instruments into one program (ALDEA) organized by stages in firms’ life cycles. Colciencias, a government agency responsible for many STI programs, has established a committee charged with adopting good practices in instrument design. The number of instruments has been reduced from approximately 60 in 2010–2014 to approximately 20 in 2019. To support agencies in estimating costs, an efficiency analysis provided capacity building for estimating programs’ economic costs from design to implementation.
· The methodology has proven scalable across regions and countries: The success of the collaboration between the WBG and Colombia’s DNP has led to new instruments assessing entrepreneurship, science, technology, and innovation programs in several countries in other regions and at other income levels, including in Latin America and the Caribbean, East Europe and Central Asia, South Asia, South East Asia, and Sub-Saharan Africa, including some beneficiaries of the International Development Association. These adoptions have varied by scope of program (e.g., a focus on evaluating SME support policies) and levels of government implementation (national and subnational). As countries grow increasingly concerned with leveraging entrepreneurship and innovation as drivers of productivity growth, and as our engagements enable us to build a rich database, the WB expects that cross-country comparisons will help clients broaden their perspective on what is possible when it comes to formulating and advancing effective entrepreneurship and innovation policy.
Bank Group Contribution and Partnerships
Colombia’s successful implementation of PER for entrepreneurship, SMEs, technology, and innovation was possible because the World Bank and its counterparts were strongly committed to the process. The partnership included DNP and several other agencies, such as Colciencias, the National Training Service (Servicio Nacional de Aprendizaje, SENA), iNNpulsa, and many other ministries and departmental governments. By providing access to information collected to manage the programs, the WB worked together with clients to determine how to make the best use of this information and to generate systematic monitoring indicators that can be used to enhance programs and investment in general.
Beneficiaries
The rationalization of the policy mix to support STI and entrepreneurship can lead to significant direct and indirect benefits. In Colombia, to take only one example, the program supporting scholarship for PhD students domestically and abroad has more than 3,000 beneficiaries. Thousands of firms benefit from the other 128 programs, but more importantly, many firms and researchers not yet benefiting from Colombia’s STI programs may become beneficiaries following further improvements in the programs’ design, implementation, and governance.
In addition, adoption of the portfolio framework has strengthened STI policy practice at the subnational level, with higher uptake, more balanced risks, economies of scale, and common development goals. More importantly, adaptation of the portfolio concept for planning public investment will lead to measurable efficiency gains in the investment of Colombia’s national budget.
