00:06 Our keynote speaker up here,
00:08 and I think you're going to,
00:11 uh,
00:12 when this is all over,
00:14 when you've heard what he has to say,
00:15 you're gonna feel that you,
00:16 uh,
00:17 were lucky to be here today
00:19 to,
00:19 uh,
00:19 hear the thoughts of someone who is,
00:23 uh,
00:23 uh,
00:23 such a,
00:25 such a renowned
00:27 person in his field.
00:28 But you know,
00:28 he's not just any kind of renowned economist.
00:32 Um,
00:33 he is credited with transforming global research in development economics,
00:38 a field that he's been pursuing for many,
00:41 many years.
00:42 In fact,
00:43 uh,
00:45 in his early years as a student
00:46 at Harvard,
00:48 he wrote his senior thesis
00:50 comparing an employment guarantee program in India
00:54 with a food subsidy program in Sri Lanka.
00:57 So,
00:58 There you go,
00:59 uh,
00:59 when other people are writing about Lord knows what,
01:01 he was already looking at this important field
01:04 where he has continued to grow,
01:07 continued to develop,
01:08 and I'm talking about Professor Michael Kramer.
01:12 Uh,
01:12 he is at the University of Chicago Development Economics Program.
01:16 He's had a long,
01:16 illustrious career,
01:18 so many different places,
01:19 uh,
01:20 developing this field,
01:21 and so much so
01:23 that in 2019,
01:24 he co-won with two other economists,
01:27 uh,
01:27 the Nobel Prize in Economics.
01:30 Uh,
01:30 he is,
01:31 uh,
01:32 they were recognized for their,
01:33 uh,
01:34 experimental approach to alleviating global poverty.
01:38 This
01:38 groundbreak.
01:39 Breaking research has reshaped our understanding of the role of human capital,
01:43 poverty alleviation in developing countries.
01:46 Um,
01:46 there's,
01:47 you know,
01:47 if you look at his wiki page,
01:48 he,
01:49 I could go on and on,
01:49 but instead,
01:50 I'm going to now bring you Michael Kramer to the podium.
01:54 Michael,
01:54 welcome.
01:59 OK.
02:07 Great.
02:07 Thank you very much for that,
02:08 uh,
02:09 for that,
02:09 uh,
02:10 nice introduction.
02:11 Um,
02:12 So
02:13 earlier speakers noted the gap
02:16 uh between the ambitious targets that the world has set for itself on development,
02:21 on climate change,
02:23 um,
02:23 and the resources that have been made available and are likely to be made available
02:28 uh for that,
02:29 for those purposes.
02:30 Um,
02:31 and you know that naturally raises the question of how can we produce more
02:36 output
02:37 with,
02:38 uh,
02:38 with,
02:39 with a given input.
02:40 And
02:41 that's a classic question in economics and
02:44 how do we move out the production function
02:46 and in fact,
02:47 producing more with given resources
02:51 is technically the definition of innovation in economics
02:54 and so that's really
02:56 a topic that I,
02:57 I'd like to focus on.
02:59 Um,
03:00 let me just note
03:01 that
03:02 while I'm,
03:03 I'm going to talk about
03:05 um
03:06 The need for innovation and the role that innovation can
03:09 play in helping us get more out of resources,
03:11 you know,
03:12 that doesn't conflict at all with the idea that
03:14 more resources should be devoted to these important goals
03:19 to match the ambitions that we've set for ourselves.
03:22 And in fact,
03:23 you know,
03:23 I think I would argue that
03:27 To the extent that we can show that we can deliver more for resources,
03:30 that actually is very complimentary
03:32 uh with that goal.
03:33 So I,
03:34 I don't wanna
03:35 I think
03:36 sort of create a false opposition.
03:39 So I'll discuss a few individual innovations,
03:41 um,
03:43 but
03:43 beyond individual innovations,
03:45 I think we need to innovate on the system
03:47 that we have for promoting innovation and development.
03:52 You know,
03:52 we can often,
03:53 it's natural to take certain
03:56 institutional features of our system as constraints.
03:59 Um,
03:59 so for example,
04:00 most World Bank funds are lent to national
04:03 governments and national governments have to repay those loans
04:06 and they will appropriately think about,
04:09 you know,
04:09 is this a good loan for them to take.
04:12 Um,
04:13 but
04:13 you know there are some
04:14 investments,
04:15 uh,
04:16 particularly.
04:16 Investments in innovation
04:18 which can benefit many countries
04:20 and our international or even global public goods.
04:24 I know there's a lot of debate about
04:27 is it appropriate to take funds meant for development and
04:30 shift them to certain types of global public goods,
04:33 but within global public goods,
04:35 there's certainly a subset
04:36 that primarily are relevant for low and middle income countries,
04:40 for example,
04:41 Certain
04:42 health problems are most common in low and middle income countries.
04:45 Certain agricultural problems.
04:47 I don't think it's at all limited to those two sectors,
04:50 but
04:50 that type of global public investing,
04:53 I'll argue that investing in that type of global public good
04:56 does
04:56 offer an opportunity
04:59 to deliver more outcomes
05:01 for for
05:02 for a given amount of investment.
05:04 And that I think is.
05:06 If to the extent that our
05:08 current institutions aren't set up well to enable that,
05:11 maybe it's worth considering reforms
05:13 to some of our institutions for that purpose.
05:16 Um,
05:17 OK.
05:18 Um,
05:19 so let me just start with some things that I think are,
05:22 on the one hand,
05:23 probably almost everybody in this room knows at some level.
05:27 On the other hand,
05:28 the
05:28 magnitude of these effects are so enormous that I think it's worth,
05:32 uh,
05:33 you know,
05:33 periodically reminding ourselves of them.
05:36 So first,
05:37 let me just note the
05:38 tremendous impact of innovation
05:41 and
05:42 helping us achieve more with the,
05:44 with uh,
05:45 with given resources
05:46 in one area that's a focus of a lot of goals,
05:49 which is
05:50 uh child survival.
05:53 So this is,
05:53 uh,
05:54 what this graph shows
05:55 is
05:56 the top shows uh is a graph of health and income in 1900.
06:01 And so you see Italy there,
06:03 and in 1900,
06:05 300 out of every 1000 children
06:08 born in Italy died before their fifth birthday.
06:12 Now,
06:13 the bottom part of that shows the relationship in today,
06:17 uh,
06:17 today,
06:18 and I've highlighted Kenya.
06:20 In Kenya,
06:21 less than 5% of children die before their 5th birthday.
06:25 So,
06:25 you know,
06:26 huge improvement in human welfare,
06:28 almost unfathomable improvement in,
06:30 in human welfare.
06:32 And yet,
06:33 Is that because
06:34 Kenya got is today is richer than Italy?
06:37 No,
06:37 Kenya is actually,
06:38 Kenya today is actually slightly lower income than Italy in 1900.
06:43 The difference is a whole suite of innovations that enabled that,
06:47 and
06:48 you know that includes everything from a bunch of technological innovations,
06:52 development of antibiotics,
06:54 vaccines,
06:55 oral rehydration therapy,
06:57 but also social innovations
06:59 that provide the vast majority of women and
07:01 their children with access to prenatal care,
07:04 um,
07:04 attended delivery,
07:05 and essential healthcare.
07:07 I think there's lots of potential for further innovation,
07:10 um,
07:12 but,
07:12 you know,
07:13 while there's strong incentives for certain types of innovation,
07:16 um,
07:17 and,
07:18 you know,
07:18 purely exist
07:20 commercial incentives under our existing institutions
07:23 are somewhat weaker for other part,
07:25 other types of innovation that are socially very valuable.
07:29 Uh,
07:29 so within health,
07:31 you can think about a malaria vaccine,
07:33 for example.
07:34 Or
07:34 uh low cost water treatment for,
07:36 for rural areas.
07:38 I'll come back to the
07:40 discussion later of uh
07:42 of pneumococcal vaccine.
07:43 Um,
07:44 there's uh,
07:45 there are different strains of pneumococcus in high and
07:48 middle income countries and in lower income countries,
07:50 and vaccines were developed first,
07:52 as you can imagine,
07:53 for those in higher and middle income countries.
07:57 Let me,
07:57 I think economic theory can shed light on when there'll be
08:01 relatively large gaps and when there'll be smaller gaps between
08:05 commercial incentives and social incentives.
08:08 So,
08:08 uh,
08:09 you know,
08:09 there are,
08:09 there are multiple cases.
08:10 This is an,
08:11 you know,
08:11 very interesting area in economics,
08:13 uh,
08:13 very rich and complex,
08:15 but let me just give a couple of,
08:17 you know,
08:17 fairly straightforward examples.
08:19 One is the,
08:20 one is the case of uh
08:22 of,
08:23 of uh externalities.
08:25 So
08:26 some innovations create benefits,
08:27 not just for the user of the good,
08:29 but for other people.
08:31 Um,
08:32 so a good example of that would be an industrial innovation
08:35 or an agricultural innovation for that matter,
08:37 that reduces greenhouse gas emissions.
08:39 Um.
08:41 So I'll come to the case of green cement,
08:43 but cement produces a lot of emissions.
08:46 If we found a way of producing cement that released fewer emissions,
08:50 well,
08:50 that would be of enormous benefit to the world,
08:52 but it's not clear that
08:55 the buyers of cement would pay more for that.
08:57 So there might be insufficient incentives to invest in that.
09:01 Another case is
09:03 innovations in provision of government services,
09:06 um,
09:07 such as improvements in tax administration,
09:09 for example.
09:10 You know,
09:10 if I think about,
09:11 you know,
09:12 we may not think of it,
09:13 uh,
09:14 today as one,
09:14 but value-added taxes were not common in most of the world.
09:18 Oh,
09:18 if we went further back and thanks.
09:20 In part to,
09:21 I don't know whether I'm allowed to say nice things about the IMF here,
09:24 but,
09:24 uh,
09:25 thanks in part to the uh to the,
09:27 the work of the IMF,
09:28 um,
09:29 you know,
09:29 these are now
09:29 common around the world,
09:30 and I think they,
09:31 they are an efficient way of raising,
09:33 raising taxes.
09:34 Um,
09:35 but you can also think about ways of provision,
09:37 providing health,
09:38 agricultural extension,
09:39 um,
09:40 you know,
09:41 education.
09:43 So I think there's a role and
09:44 it's difficult for firms to,
09:47 to make
09:49 money anywhere near,
09:50 I'm not saying you can make zero money,
09:52 but it's hard to get
09:53 for them to,
09:54 they don't have this,
09:55 the incentives that are commensurate with the social value of,
09:58 of some of these innovations.
10:00 Um,
10:00 I'd also note that some of these are global public goods as well,
10:04 and that creates a little bit of a free rider problem among countries.
10:08 Um.
10:09 So
10:10 if we look,
10:10 I've been talking,
10:12 given a few examples,
10:13 if you look at studies of the rate of return to innovation,
10:16 there are a number of them,
10:17 and they generate
10:18 very high estimates of the rate of return,
10:20 things like 60,
10:21 70%
10:23 annual rates of return,
10:24 uh,
10:24 social return.
10:25 Um,
10:26 so
10:27 I will,
10:28 I should note,
10:29 however,
10:30 that
10:31 innovation is first inherently risky,
10:33 and second,
10:34 it's very easy
10:35 to throw good money after bad.
10:38 And
10:39 When we think about institutions to encourage innovation,
10:43 we need things that will,
10:44 um,
10:45 you know,
10:45 accelerate the pace of innovation,
10:47 shape it to meet social needs,
10:50 but also we need to design institutions
10:52 that can limit some of the waste that might come if we're,
10:55 if we're,
10:56 if we're,
10:57 if we,
10:57 if we don't find a way of preventing throwing good money after bad.
11:01 So let me give a,
11:02 you know,
11:02 say a little bit about the rest of the talk.
11:04 I've been working on,
11:06 uh,
11:06 I've been chairing a commission on innovation for climate change,
11:09 food security,
11:10 and agriculture.
11:11 Um,
11:12 so I've been thinking about innovations around climate adaptation,
11:16 to some extent mitigation
11:17 in,
11:18 in around food security and agriculture.
11:20 So I'll say,
11:21 I'll
11:22 just as.
11:22 Examples,
11:23 I'll choose examples from that sector
11:25 and
11:25 sort of make the case that not only there's historical examples,
11:28 but there are things we could be doing
11:30 right now.
11:31 Um,
11:32 um,
11:33 second,
11:33 I'll discuss,
11:34 oh I see two ones there anyway,
11:36 um,
11:36 the,
11:36 um,
11:37 um,
11:38 uh,
11:39 institutions for,
11:39 for,
11:40 uh,
11:40 for innovation,
11:41 and I'll talk about two types of institutions.
11:43 OK.
11:46 OK,
11:47 so some
11:48 some innovations for climate adaptation.
11:51 So with climate change,
11:52 not only are we're seeing
11:54 higher temperatures,
11:55 for example,
11:55 but we're seeing increased uncertainty around weather.
11:58 And
11:59 um,
12:00 you know,
12:00 variation in daily precipitation is estimated to increase by about 5%
12:05 and temperature variability increases by
12:07 10 to 15% per.
12:09 Degrees Celsius of warming.
12:11 So
12:11 this is,
12:13 um,
12:13 and there's also
12:15 increased variation in the timing of agricultural seasons.
12:18 So if you think about a farmer who's used to
12:20 traditional times of year to plant for other operations,
12:24 certain rules of thumb,
12:25 those rules of thumb may not work anymore.
12:28 Now,
12:29 there's
12:30 a lot of technological progress on this,
12:32 so
12:33 it's now possible to predict rainy season or monsoon onset
12:38 up to a month in advance in,
12:39 in,
12:40 in many parts of the world,
12:42 um,
12:43 and,
12:43 you know,
12:44 AI is enabling
12:46 very,
12:46 very rapid advance in,
12:48 in some areas of weather forecasting.
12:50 And it allows customized weather forecasts to be made,
12:53 so it's not just a general weather forecast.
12:56 You know,
12:56 this is the probability of rain today,
12:58 probability tomorrow.
12:59 A farmer might need to know very specific information,
13:02 you know,
13:03 is it going to,
13:03 what's the prob,
13:04 what's the
13:05 probability of rain in the next 3 days?
13:07 Uh,
13:08 what's the,
13:08 or something much more complicated than that?
13:10 Should I
13:11 plan today or is it better to
13:13 take the option of waiting after that?
13:15 Well,
13:15 it's now.
13:16 You can set AI,
13:19 you know,
13:19 AI models can be optimized.
13:21 The,
13:21 the loss function on them in technical terms can be
13:24 set around whatever parameter is most important to the,
13:27 to the farmer,
13:28 and these can be run very quickly.
13:29 The big firms in this area like Nvidia and so on,
13:32 they have open access models that can easily be tweaked.
13:36 Talking to a researcher,
13:37 a colleague said,
13:39 you know,
13:39 a grad student or a postdoc can do this in a,
13:42 in a day.
13:43 Um,
13:44 the,
13:45 um,
13:46 now there's
13:47 a lot of evidence from multiple studies
13:49 that farmers adjust their behavior,
13:51 their agricultural behavior,
13:52 and their investment decisions in response to accurate weather forecasts.
13:56 But
13:57 nonetheless,
13:58 most farmers are not getting this information right now.
14:01 And
14:02 um you know,
14:03 there's a couple of RCTs that estimate benefits.
14:06 I don't want to claim it's just everywhere,
14:08 but a couple of RCTs,
14:09 randomized controlled trials,
14:12 find benefits over $100 per farmer,
14:14 and
14:15 the cost of,
14:16 of,
14:17 of
14:17 producing
14:18 these forecasts and even disseminating them,
14:21 you know.
14:22 Maybe 1/100 of that.
14:25 Um,
14:25 so,
14:26 um.
14:27 In part because they can be delivered digitally.
14:30 Um,
14:31 so
14:32 we've been working with the Indian government.
14:35 They just for the most recent monsoon,
14:38 um,
14:38 they delivered,
14:39 uh,
14:39 forecasts of monsoon onset and total rainfall in certain parts of India,
14:44 not,
14:44 uh,
14:44 not Mumbai,
14:45 which that wasn't
14:46 part of this pilot year,
14:48 um,
14:48 um,
14:49 uh,
14:49 4 to 9 million farmers.
14:51 Um,
14:52 and you know,
14:53 the government of India has a list of 100 million phone numbers,
14:56 uh,
14:57 farmers,
14:57 so it
14:58 should be very easy.
14:59 To expand this
15:00 and of course other countries in South Asia,
15:03 Africa,
15:04 in fact,
15:04 other areas,
15:05 Latin uh
15:06 Inter-American Development Bank to speak well of another nearby institution,
15:10 you know,
15:10 they're,
15:11 they're working with,
15:12 uh,
15:12 they're,
15:13 they're trying to extend this in Latin America.
15:16 Um,
15:16 now,
15:17 I don't want to claim this is all automatic,
15:19 you know,
15:19 things need,
15:20 when you're doing this,
15:21 you need to think through how do you
15:23 best communicate with farmers about probabilistic information,
15:26 um,
15:27 how do you build farmer trust.
15:29 You probably want to build in a bunch of A B tests to optimize these types of systems,
15:34 but the reason,
15:35 what time of even boring things,
15:36 what time of day do you send the messages to the farmer?
15:39 How many repetitions?
15:40 Well,
15:41 that's tech companies are doing A B tests like this all the time.
15:45 I think we could be,
15:46 we could be doing similar things,
15:47 and the benefits will exceed
15:49 beyond any one country.
15:51 OK.
15:52 OK,
15:53 let me turn to another,
15:54 you know,
15:54 that's sort of a technological innovation that also needs a bunch of
15:58 uh institutional and social adaptation.
16:00 Here's something that's
16:02 more
16:04 Not particularly technological at all.
16:06 It's just a
16:07 um a financial market innovation.
16:10 So,
16:10 obviously a vast literature on microfinance,
16:13 and
16:13 I think uh
16:14 that the latest wave is a little bit of an uptake,
16:17 uptake on that.
16:18 There was a lot of excitement,
16:20 and a lot of disillusionment,
16:21 but there's,
16:22 uh,
16:22 you know,
16:22 still some,
16:23 I,
16:23 I think people,
16:24 we don't want to throw out the baby with the bathwater on that.
16:27 But,
16:27 you know,
16:27 one area that's been somewhat disappointing is the effect on
16:31 productive investments.
16:33 Well,
16:33 one of the interesting areas of research that,
16:36 you know,
16:36 many people are involved in
16:37 is can we try to adjust the,
16:40 the
16:40 typical microfinance contract to make it
16:42 more conducive to expanding actual investment.
16:46 So here's a study that,
16:48 uh,
16:48 that I was involved in with uh Tawni Surrey and,
16:51 and,
16:51 uh,
16:52 and Billy Jack and Yost a lot.
16:54 Um,
16:54 this,
16:55 you see a picture here of a,
16:56 of a rainwater harvesting tank.
16:58 Dairy cattle need a lot of,
16:59 of water.
17:00 They need to be water every day.
17:02 And of course,
17:02 humans need water.
17:03 Um,
17:04 so,
17:04 um,
17:05 these,
17:05 these tanks,
17:06 you see them everywhere on top of roofs in urban areas,
17:09 but they can also be useful in agriculture.
17:12 They cost,
17:12 um,
17:13 but they cost $300
17:15 the 5000 L tank.
17:17 That's not,
17:18 that's not trivial for a,
17:19 for a farming family in Kenya to buy.
17:22 So we looked at just
17:24 You can,
17:25 if you take a look at that,
17:25 there's no moving parts.
17:26 It's pretty big,
17:27 hard to hide,
17:28 makes pretty good collateral.
17:30 So we worked with a savings and loan cooperative that tried
17:33 saying,
17:33 OK,
17:34 we'll just do an asset collateralized loan.
17:36 They got takeup of the loan product went from 2%
17:39 to 25%
17:42 without any increase in default.
17:43 Rates
17:44 and dairy farmers were producing more milk.
17:46 They were also better able to to get through periods when the rain was
17:50 was not coming.
17:51 And they,
17:51 they actually,
17:52 this was a study,
17:53 but the,
17:54 the dairy cooperative and the Associated Savings and Loan
17:56 Association continued the program afterwards with their own funds,
18:00 and other cooperatives are doing that.
18:02 This is another thing that I think,
18:04 you know,
18:05 you could invest in scale,
18:06 and here I think the gap between commercial and social incentives are much,
18:10 much smaller.
18:11 But if you think about each stage of taking this on a little bit more broadly,
18:15 you know,
18:16 it's another thing for a financial,
18:18 for a bank or some other financial provider
18:21 to expand access to this program,
18:22 working with savings and credit cooperatives or dairy cooperatives.
18:26 They would be taking some risks.
18:29 If they succeed in that,
18:31 a bunch of other
18:33 financial institutions are going to enter that market,
18:35 and they're not going to get a monopoly on this market.
18:37 They're going to have to share the profits in that.
18:39 If they fail,
18:40 they're going to take the losses.
18:42 So this is an area where,
18:44 to the extent that there is an institution that
18:47 is willing to provide
18:49 capital for climate resilience or try or willing to
18:53 willing to invest in
18:56 in.
18:56 Yeah,
18:56 these are also particularly favorable to,
18:58 to women because
19:00 women are often enough to do a lot of the
19:02 work of collecting water,
19:03 um,
19:03 that has that as a social objective,
19:05 it's worth
19:06 probably helping finance at least experimentation on that,
19:09 even if not,
19:10 you know,
19:10 not necessarily long run subsidies.
19:13 OK,
19:14 um.
19:16 OK.
19:17 So those are examples of uh,
19:18 of,
19:19 of types of innovation that um
19:21 that can be used,
19:22 and there's a long list just within agriculture,
19:25 food security,
19:26 climate change,
19:27 that we've got on our website for the commission,
19:29 we've got a,
19:29 a long,
19:30 a long list of,
19:31 of innovations with
19:33 evidence of impact and cost effectiveness that we think are,
19:36 are ready to scale.
19:37 Um.
19:39 Let me try to take a,
19:40 uh,
19:41 a little bit more of a,
19:43 of a systematic approach to this and
19:46 also
19:47 transition to
19:48 what I call meta-innovations,
19:50 innovations in the system for creating innovations.
19:53 Um,
19:54 so I'll talk about two things here.
19:56 Let's first talk about
19:58 open-tiered
19:59 evidence-based uh social innovation funds.
20:02 So that I'm involved with,
20:03 uh,
20:04 um,
20:04 um,
20:05 Development Innovation Ventures,
20:06 which is a part of USAI.
20:08 Um,
20:09 FID in France,
20:10 uh,
20:11 uh,
20:11 has a lot,
20:12 a lot of similarities.
20:14 Both of these are designed to be quite open to applications,
20:18 um,
20:18 and not necessarily top-down,
20:20 open to ideas,
20:22 um,
20:22 across sectors,
20:24 across countries,
20:25 both open to commercial scaling and to public scaling,
20:29 um,
20:29 but the,
20:30 you know,
20:30 how do you discipline that openness and how
20:32 do you avoid throwing good money after bad?
20:35 Well,
20:35 we have pretty strict rules about shared funding.
20:38 So there are very small grants of,
20:39 of,
20:40 of,
20:40 of,
20:40 uh,
20:41 for piloting.
20:42 But then to get more serious amounts of money,
20:44 um,
20:45 it has to go undergo rigorous testing for impact
20:49 and cost effectiveness
20:51 if it's something that would be scaled with public sector funding,
20:53 slightly different rules if it would be scaled uh through the private sector.
20:57 And then,
20:58 um,
20:58 and,
20:59 and then there's,
21:00 so we have funds available for that and then we have funds for
21:03 transitioning this to scale because,
21:05 That,
21:06 you know,
21:07 additional
21:07 tweaks are needed for that,
21:09 uh,
21:09 more than tweaks to adapt things for scale.
21:12 So me,
21:13 we've now been around for a bit over a decade,
21:16 uh,
21:16 um,
21:17 and
21:18 we can review the evidence from our full portfolio to
21:21 look at what's the rate of return to these investments
21:24 and maybe try to draw some lessons about which innovation scale.
21:28 Um,
21:29 so let me share some of this.
21:31 First,
21:32 here's a picture
21:33 of
21:34 our 1st 42 grants,
21:35 we took the 1st 2 years of our portfolio.
21:38 How much did uh different innovations scale?
21:40 And what you can see here is very skewed distribution.
21:44 And of course,
21:44 that's extremely typical of any type of innovation fund.
21:47 If you look at patent citations,
21:49 if you look at citations to academic articles,
21:52 you'll see exactly this.
21:53 And any venture capital fund knows,
21:55 you know,
21:56 they're not going to get hit with everything,
21:57 but if they invest in Google or Facebook,
22:00 then that's,
22:00 then they're gonna be
22:01 uh very successful.
22:03 In fact,
22:03 um,
22:04 um,
22:05 so
22:06 the,
22:06 so what are the,
22:07 what are some characteristics of things that scale?
22:11 Well,
22:11 we broke this down by
22:13 uh award stage.
22:15 And
22:16 you know,
22:17 there's a saying in development that pilots never scale,
22:20 so we thought we'd look at that empirically.
22:23 And indeed,
22:24 you know,
22:24 there's only a 17 scaling here is reaching a million people or more,
22:28 uh,
22:28 that's somewhat arbitrary how you count that.
22:30 I'll I'll come back to,
22:32 oh,
22:32 I think it is a better metric overall,
22:34 but
22:34 17% of the stage ones reached over a million people.
22:38 The stage threes
22:40 that,
22:41 you know,
22:41 there was only one of those,
22:42 and it succeeded.
22:44 you might say,
22:45 OK,
22:45 maybe the pilots aren't working out so well,
22:47 but here's a different way of looking at it.
22:50 Here's the
22:51 award value
22:52 and the number of people reached,
22:54 and the people reached per dollar,
22:55 and you get a totally different picture.
22:57 I don't want to play this up in
22:59 terms of statistically significant differences and so on.
23:02 They're not,
23:02 you know,
23:02 these are small samples and.
23:04 Uh,
23:05 not,
23:05 not,
23:05 uh,
23:05 causal estimates for those of you are,
23:08 you know,
23:08 who are,
23:08 uh,
23:08 technical here,
23:09 but,
23:10 um,
23:10 the,
23:11 you know,
23:11 I think the lesson to take away from this
23:13 is we shouldn't write off investing in pilots,
23:16 um,
23:17 and you're,
23:19 at least with this data,
23:20 you're actually reaching more person,
23:22 more people per dollar spent with that.
23:24 Um,
23:25 now.
23:27 You know,
23:27 what other lessons?
23:29 Well,
23:29 a huge thing was cost.
23:31 You know,
23:32 we,
23:32 we invested in innovations for low and middle income countries.
23:36 So
23:37 things that cost less than $3 per person
23:39 based on the documents when they came in,
23:41 they were much more likely to scale.
23:43 It
23:44 perhaps correlated with,
23:46 and I should note there's sort of
23:48 this
23:48 works both ways.
23:49 Once things scale,
23:51 unit costs go way down.
23:53 Um,
23:53 I think that's an insufficiently appreciated
23:56 point.
23:56 There's very good work by the International Rescue Committee.
23:59 Uh,
23:59 they look at the cost of,
24:01 of
24:02 cash transfers,
24:03 and you know,
24:04 some programs within the IRC,
24:06 they pay more than,
24:07 spend more than $1 to deliver.
24:09 A $1 to a recipient.
24:11 Others they spend 14 cents,
24:12 and the biggest determinant is the scale at which they operate the program.
24:16 So,
24:16 um,
24:16 but in any case,
24:17 interventions that have low cost are much more likely to scale.
24:21 Here's a
24:22 perhaps a
24:23 related implication.
24:25 You know,
24:26 when you're talking about an inexpensive product,
24:28 customer acquisition costs loom huge.
24:31 So
24:32 if you're,
24:33 you know,
24:33 a lot of social entrepreneurs like other entrepreneurs
24:36 think first of selling to the customer,
24:38 to an ultimate customer,
24:39 but actually
24:40 a lot of businesses sell to other businesses,
24:42 they sell to governments,
24:44 those that already have the network
24:46 by having a strategy of,
24:48 of going to
24:50 people who already have a network for reaching the ultimate customers,
24:53 they are much more likely to scale.
24:55 And then finally,
24:56 and obviously I'm a biased party on this,
24:58 um,
24:59 but
25:00 having some
25:01 rigorous evidence of,
25:03 of impact
25:04 uh through a randomized controlled trial,
25:06 that was,
25:07 a lot of people think of that as slowing things down,
25:10 as inconsistent with lean innovation.
25:13 We actually see in our data
25:15 that the proposals that came in with that were much more likely to scale
25:19 than the ones that didn't,
25:20 and that's very correlated with having a
25:22 a development economics researcher in there.
25:25 I think that,
25:26 you know,
25:26 there are many stories about why that relationship might be there,
25:30 but I'll speculate that,
25:31 You know,
25:32 in a,
25:32 in a lot of,
25:33 in a lot of fields,
25:35 it's
25:36 considered very normal for researchers to get
25:38 involved in developing products and certainly in,
25:41 in computer science and,
25:43 in,
25:43 uh,
25:43 in biotech.
25:45 I think there are some
25:47 areas within economics,
25:49 actually,
25:49 you know,
25:50 um,
25:50 economic theory,
25:51 there are,
25:52 there are researchers who get very involved with auction design,
25:55 etc.
25:56 I think
25:57 some areas of,
25:58 of development economics are now starting to,
26:01 to be such an area,
26:02 um,
26:03 and particularly uh um
26:05 areas that involve randomized controlled trials.
26:08 Um,
26:08 they
26:09 bring the researcher very close to the problem that they're,
26:11 that the researcher is studying,
26:13 get perspectives from the customer from a variety of different places.
26:17 Um,
26:18 anyway,
26:18 that's a separate discussion,
26:19 but empirically that doesn't seem to be a negative and if anything
26:23 seems to be a pretty strong positive.
26:26 OK,
26:26 so
26:28 I mentioned,
26:28 you know,
26:28 you can count the number of users different ways,
26:31 um,
26:32 but we did try to calculate the benefits.
26:34 If you remember this chart back here,
26:36 um,
26:37 the things,
26:38 most of these we couldn't quantify.
26:40 They weren't,
26:40 couldn't express them in dollar values,
26:43 didn't have a,
26:43 have a good RCT evidence,
26:45 but the ones in red,
26:46 we could.
26:47 So for 5 out of these 43,
26:50 we,
26:50 we could put a dollar value on the benefits.
26:53 And we add up that.
26:54 Dollar value.
26:55 Our total portfolio,
26:56 our total investment was $16 million over this period.
27:00 If we look at the total benefits,
27:02 $260 million
27:04 so 17 times as big.
27:05 And um
27:07 um
27:07 so
27:08 I think that suggests that this is an area with very high returns.
27:12 You might then say,
27:13 well,
27:14 how can there be high returns?
27:15 We don't think that government investment programs
27:18 are,
27:18 you know,
27:18 governments are good at picking winners.
27:21 I think there are
27:22 really two reasons for that.
27:23 I would,
27:24 I would speculate.
27:25 One is that
27:27 these are social returns,
27:28 these are not necessarily private returns.
27:30 So the biggest winner,
27:32 so to speak,
27:33 was
27:33 a software firm in Cambridge,
27:35 Massachusetts that developed software for community health workers.
27:39 That got,
27:39 that reached,
27:40 you know,
27:41 Indian,
27:41 that was used for Indian community health workers,
27:44 reached 60 million clients,
27:47 mothers.
27:48 There's strong evidence of impact,
27:50 health impact,
27:51 but did they make much money?
27:53 No,
27:54 for two reasons.
27:55 First,
27:56 um,
27:56 you know,
27:56 I don't think they were being paid that much for it.
27:59 Second,
28:00 the Indian government.
28:01 Understandably eventually went with their own supplier
28:04 and they they lost this business,
28:06 but the Indian government's continuing to
28:08 get software for community health workers.
28:10 It's just not
28:11 now if you're a private investor,
28:13 you know,
28:13 Warren Buffett talks about
28:15 you want a moat,
28:16 you know,
28:16 you want some first mover advantage for the things you invest in.
28:20 If you're a public investor.
28:22 If you're,
28:22 you know,
28:23 we're part of the US government promoting development,
28:25 if others are going to come in and imitate,
28:27 then that generates more benefits for society.
28:30 That's not necessarily a negative,
28:31 it's actually in some ways a positive.
28:34 OK.
28:34 Um,
28:35 the other thing,
28:36 so that's why I think it's possible,
28:38 why not all the great ideas will be scooped up by the private sector.
28:41 Some of them are
28:42 good for society,
28:43 but not necessarily commercially good.
28:45 But the other element,
28:46 and here this is speculation,
28:47 is,
28:48 I think the discipline of saying we don't invest large amounts
28:50 of money until there's rigorous evidence of impact and cost effectiveness,
28:55 I think that was important in avoiding some,
28:57 some wasteful investments.
29:00 OK,
29:01 we talked about,
29:02 you know,
29:02 one,
29:03 final,
29:03 uh,
29:04 uh,
29:04 set of,
29:05 of things,
29:06 uh,
29:06 a
29:07 sort of a meta-innovation which is
29:09 trying directly to incentivize the private sector to harness the,
29:13 uh,
29:14 the great inventiveness they've shown in developing products for,
29:17 uh,
29:17 where there is a commercial market,
29:19 for areas that where there's social needs,
29:21 but where the market isn't as attractive.
29:24 So
29:25 intellectual property systems,
29:27 they reward innovators with monopoly rights for some period of time.
29:31 That's great in that it focuses attention on things people are willing to pay for,
29:37 but a downside
29:38 is that
29:39 there's a monopoly,
29:40 and that creates,
29:41 in the bloodless language of economics,
29:43 static losses.
29:44 But in the,
29:45 in some cases,
29:46 you know,
29:47 that means fewer people might get access to life saving drugs or.
29:50 Scenes,
29:51 uh,
29:51 so we might be worried about it.
29:52 And there's
29:53 interminable debates and shifts back and forth on intellectual property rights.
29:58 And I think that's because the ins
30:00 working within the current institutional framework,
30:03 sort of,
30:04 do you want more of one and less of the other,
30:05 or do you want to switch it?
30:07 These are two really important goals,
30:08 access and,
30:09 and incentives for R&D.
30:11 Well,
30:11 we can break that,
30:12 that whole paradigm
30:13 if we just expand the set of institutions.
30:17 And
30:17 I think there's opportunities to simultaneously
30:20 incentivize R&D and improve access.
30:22 So let me
30:23 talk about one effort that I was involved in.
30:26 Which is something called an advanced market commitment.
30:29 So those are legal commitments made by governments or by donors,
30:33 um,
30:33 to pay
30:34 uh a top up,
30:36 um,
30:36 for,
30:37 uh,
30:38 in this case a vaccine meeting certain uh
30:40 uh specifications
30:42 if the governments are,
30:43 are want to want to
30:45 pay something for it,
30:46 and if the firm agrees to some uh maximum price.
30:51 That basically
30:53 offers the opportunity by committing
30:56 to buy this and to buy a large quantity
30:58 that offers the opportunity,
30:59 the firm an opportunity to go from a
31:02 relatively high priced market serving a small number of people
31:06 to
31:07 much higher volumes at a somewhat lower unit price.
31:11 That can increase their total incentives and it can,
31:13 can get access.
31:15 So an advance market commitment was set out for pneumococcus,
31:18 um,
31:18 $1.5 billion was committed.
31:21 Um,
31:21 there's a,
31:22 um,
31:23 these were the prices that were chosen.
31:25 Here's some evidence on,
31:28 uh,
31:28 so three vaccines were developed against pneumococcus,
31:31 two by,
31:32 by large multinational firms,
31:34 a third recently developed in India,
31:36 um.
31:38 This was,
31:39 um,
31:39 you know,
31:40 this was rolled out
31:42 very,
31:42 very quickly relative to most vaccines,
31:45 um,
31:45 I think in part because of the structure of the advanced market commitment.
31:49 So I've shown a comparison between the pneumococcus vaccine
31:52 and the rotavirus vaccine which didn't have one.
31:55 It's estimated that
31:56 700,000 lives have been saved by the pneumococcus vaccine,
31:59 so,
32:00 um,
32:00 this,
32:01 this can,
32:01 you know,
32:02 potentially have a very large
32:03 impact,
32:04 hundreds of millions of doses have been,
32:06 have been delivered.
32:07 Um,
32:08 OK.
32:08 I think this approach could be used in other,
32:11 in other cases.
32:12 I won't go through this slide,
32:13 but,
32:14 you know,
32:14 cement.
32:15 7% of greenhouse gas emissions.
32:19 Governments consume and the US government consumes about half of cement in the US.
32:23 It's,
32:24 um,
32:25 there's,
32:25 it's very,
32:27 it's very polluting.
32:28 The value of a ton of cement
32:31 is
32:31 pretty much the same as the value of the carbon
32:34 that's emitted.
32:35 There are technologies for producing greener cement,
32:38 um,
32:39 but the social benefits of that would be great,
32:41 but the commercial incentives to invest aren't.
32:44 You know,
32:44 if governments
32:46 perhaps assisted by institutions like the World Bank could say,
32:49 hey,
32:50 we'll commit that if somebody develops a greener cement.
32:53 We will help finance the purchase of that in a
32:56 way that is tied to the reductions in carbon.
32:59 Well,
32:59 that's something that could help provide the stimulus,
33:02 and
33:03 you know,
33:03 governments often introduce regulations
33:06 after technology is developed.
33:08 If you just announce that in advance,
33:10 you might stimulate the development of the technology.
33:13 Um,
33:14 so,
33:15 you know,
33:15 overall high rate of return to innovations.
33:18 I think we should be investing more
33:21 and But I think we also should think about
33:23 potential new institutions
33:25 uh to accelerate
33:27 investment and innovation,
33:29 shape it towards human needs that
33:31 might not otherwise attract commercial investment,
33:33 and then structure those institutions
33:36 so that
33:36 as you know,
33:37 with advanced market commitments,
33:38 the payment's only made if the product is developed.
33:41 So with uh
33:42 um so that in ways that are
33:44 uh limit,
33:45 limit potential waste and limit throwing good money after bad.
33:48 Thanks very much.
33:58 Oh yeah,
33:58 I'm happy to,
33:59 happy to take a,
34:01 a,
34:01 a few questions and I'll,
34:02 maybe I'll gather them up and then uh then
34:04 try to answer at least some,
34:06 uh
34:16 Hello,
34:16 my name is Juliet Adams.
34:18 I'm also an educator,
34:19 and I'm very much interested to hear your perspective of
34:24 innovation and creativity in the education system,
34:27 especially as it pertains to the US.
34:29 Thank you.
34:31 Sorry,
34:31 especially as it pertains to what
34:33 the US,
34:34 OK.
34:35 Yeah.
34:48 I
34:49 I was wondering,
34:50 um,
34:51 have you
34:52 talked or exchanged ideas with either Jim Kim or Melinda Gates?
34:57 Uh,
34:57 your work sounds
34:59 very similar to Farsley Abed's at at Brax,
35:03 and as early as 2015,
35:05 both,
35:06 uh,
35:07 Jim,
35:07 Kim and Melinda Gates
35:10 were using that in the middle of women empowerment
35:13 and then moving that through into technology.
35:16 So I was just wondering if you'd come,
35:18 uh uh I think there might be some very strong multipliers there.
35:33 Um,
35:34 hello.
35:34 Um,
35:35 so my question is,
35:36 in a decade or two,
35:37 do you think
35:38 that,
35:39 um,
35:39 entrepreneurship and innovation
35:42 from an economics point of view would,
35:44 um,
35:45 would contribute to solving unemployment?
35:47 And if so,
35:48 to what extent
35:50 can we rely
35:51 on,
35:52 um,
35:53 entrepreneurship and innovation,
35:54 especially,
35:55 um,
35:56 uh,
35:57 tech-based,
35:57 uh,
35:58 um,
35:58 enterprises?
36:08 Uh,
36:08 hi.
36:09 Uh,
36:09 my question is,
36:10 uh,
36:11 what do you think
36:12 higher education
36:14 can do to facilitate and promote innovation,
36:17 mostly universities.
36:18 Thank you.
36:20 OK.
36:21 OK.
36:21 Uh,
36:22 OK,
36:26 great.
36:27 Uh,
36:27 I think I already have more than I can handle.
36:29 Um,
36:30 um,
36:31 let me,
36:31 uh,
36:31 first,
36:32 um,
36:33 take the,
36:33 the,
36:34 the first question at least,
36:35 uh,
36:36 on,
36:36 on education,
36:37 and maybe this will link a little bit to,
36:39 to higher education as well.
36:42 I,
36:43 there's
36:45 There's,
36:45 I think,
36:46 a lot of poten,
36:47 there's a lot of excitement in
36:50 about artificial intelligence,
36:51 to take one example,
36:52 and I think
36:53 um
36:54 there's,
36:55 I think,
36:56 you know,
36:57 some people have the view that there's a lot of uh
37:00 of
37:01 huge amount of potential for artificial intelligence to
37:05 help in education,
37:06 um,
37:06 and some people have the view that there's
37:09 a lot of hype.
37:10 Now I think both things are.
37:12 Um,
37:13 um,
37:14 so how do we,
37:15 how do you make progress in this area?
37:17 Well,
37:18 you know,
37:18 first,
37:18 why is there,
37:19 why is there a lot of potential?
37:21 A lot of children fall very far behind
37:25 where the official curriculum is.
37:27 In that type of a situation,
37:29 it's very hard to benefit from the,
37:31 from the existing curriculum,
37:34 and you know,
37:35 those,
37:35 as we'll hear in the next session,
37:36 COVID's obviously worsened that problem,
37:38 but it was there,
37:39 uh,
37:39 it was there before.
37:41 Well,
37:41 and very important principles to meet the learner where they are.
37:45 So you can develop personalized adaptive learning systems
37:49 and,
37:50 you know,
37:50 AI can be very good at that,
37:52 at trying to understand where the student is and,
37:54 and trying to,
37:55 to help them.
37:56 Um,
37:57 now,
37:58 and then trying to figure out how best to communicate with that individual learner,
38:01 for example.
38:02 And the,
38:03 and they're very encouraging results on some
38:05 of these personalized adaptive learning systems,
38:08 huge learning gains.
38:09 At the same time,
38:11 it's
38:12 far from trivial to think about how do you integrate this into
38:16 an
38:16 into a,
38:17 into an education system that isn't built around that.
38:20 So that's going to,
38:21 working on that's going to require a combination of technological innovation
38:25 and it's going to require
38:27 a lot of social innovation.
38:29 I'm working on the ground with teachers,
38:30 with ministries of education to make progress on that,
38:33 and I'll probably require
38:35 a lot of trial and error.
38:37 And
38:37 multiple approaches,
38:39 approaches that are involved working with the students,
38:41 approaches that involve
38:42 maybe helping teachers develop lesson plans,
38:45 approaches that
38:47 involve working at the level
38:48 of the,
38:49 of the headmaster of the education system to provide support to teachers.
38:53 There's a huge amount,
38:55 huge potential in this area,
38:57 and you know,
38:57 you saw that chart.
38:59 There's going to be a lot of,
39:00 of,
39:00 of,
39:00 of the skewness of the distribution.
39:02 There'll be a lot of failures in this area.
39:04 But I think the point is if you think about this from an overall portfolio
39:08 investment approach,
39:10 you can afford some failures if we can get
39:12 learning gains.
39:13 You know,
39:14 what drives growth in the,
39:15 in the end,
39:16 well,
39:17 many things drive growth,
39:18 but technological change drives growth and
39:22 capital drives growth.
39:23 Human capital.
39:24 Accounts for correctly measured accounts for a
39:27 huge fraction of overall capital investment,
39:30 and that's a big sector of total big share
39:33 of total investments in countries that have high population growth
39:37 and skewed age distributions.
39:39 We have a chance now,
39:40 you know,
39:40 we've
39:41 terrible education results in much of the world.
39:43 We have a chance to improve that.
39:45 You know,
39:45 investing and improving that seems like a,
39:48 a,
39:48 a great case of the need for,
39:50 for innovation.
39:51 Um,
39:51 what can higher education do?
39:53 Um,
39:54 you know,
39:55 certainly,
39:56 um,
39:56 I think there's,
39:57 you know,
39:58 many of the,
39:59 you know,
39:59 many of the successful innovations I,
40:01 I mentioned involved researchers,
40:04 often university-based researchers in many of this.
40:07 I,
40:07 I,
40:07 uh,
40:08 I also think that
40:11 I think that having,
40:12 uh,
40:13 I,
40:13 I guess I would add 22 elements.
40:16 One is,
40:16 I think having openness
40:18 so that they can
40:19 develop their own partnerships and and propose their own ideas,
40:23 even if they're not coming from a,
40:24 a top-down analysis that we've done at USAI about,
40:28 you know,
40:28 these are the priority sectors.
40:29 Deliberately set it up that way
40:31 and obviously USA has a lot of areas it declares priorities,
40:34 but having openness
40:36 to input from
40:37 uh from researchers I think and others,
40:39 entrepreneurs
40:40 makes a lot of sense.
40:41 Um,
40:42 the other thing is sometimes you do need bigger
40:45 you know DARPA-like projects
40:47 to focus on goals.
40:49 OK.
40:49 Um,
40:50 let me,
40:50 um,
40:51 you know,
40:52 I think,
40:52 uh,
40:52 I won't go into detail on some of the other
40:55 issues,
40:55 but I think there's
40:56 a lot of creative things that can be done on gender.
40:59 You know,
40:59 I mentioned one thing related to gender,
41:01 Gender Innovation lab,
41:02 uh,
41:03 at the World Bank,
41:04 obviously does,
41:04 you know,
41:05 amazing work.
41:06 Um,
41:07 um,
41:07 the,
41:08 um,
41:09 and the
41:09 You know,
41:09 wonderful,
41:11 you know,
41:11 really exciting things on prevention of gender-based violence and
41:15 very exciting results on that from a number of,
41:17 of academic researchers,
41:19 I think,
41:19 trying to
41:20 think about how to scale those up,
41:22 um,
41:22 that's going to require additional work to convert those to scalable products,
41:25 but
41:26 it's vital.
41:27 Um,
41:28 the,
41:28 um,
41:30 The issue of entrepreneurship and unemployment.
41:34 I think this is a really interesting and important issue one.
41:39 You know,
41:40 people have focused a lot on the negative side of technology in terms of employment.
41:44 There are a lot of people
41:46 in,
41:47 and again,
41:48 the scenario where there's,
41:49 I think,
41:49 a lot of potential and a lot of hype,
41:51 but
41:52 there's,
41:53 there are a lot of people who are not very mobile
41:56 and who can't necessarily go to where the jobs are and can't do them.
41:59 If you think of farmers,
42:01 you know,
42:01 in rain-fed agriculture,
42:03 sometimes they have a lot of time during some parts of the year.
42:05 If you think of
42:07 women,
42:07 you know,
42:08 there might be.
42:08 Jobs in the city or in certain places that women can undertake,
42:12 but they're not able to move
42:14 to take those jobs.
42:15 Well,
42:16 to the extent that
42:17 technological advance can make things more mobile,
42:20 maybe the jobs can move to where the
42:21 where the people are who can do them
42:25 and.
42:28 And I,
42:28 I think I,
42:29 maybe I'll,
42:29 I'll leave it at that,
42:30 but thank you very much.
42:36 Yeah
- add-style
- lp-body-content