00:06 I'm just going to uh turn directly to our,
00:10 uh,
00:10 two presenters.
00:11 So we're gonna have the same format as the previous um session.
00:13 So we've got,
00:14 uh,
00:14 Uh,
00:15 two of our deck colleagues who are going to kick off the discussion with,
00:18 uh,
00:18 with complimentary presentations,
00:20 and,
00:21 uh,
00:21 and then we'll have 3 panelists,
00:23 and,
00:23 uh,
00:24 I had some inside information that unfortunately one of our
00:26 panelists had a flight canceled and could not join us.
00:29 So I only have 3,
00:30 which buys us the 10 minutes that allows us to wrap up on time.
00:33 So
00:33 hopefully that will work out.
00:35 Um,
00:36 so
00:37 let me,
00:37 uh,
00:37 sorry,
00:37 who's going first,
00:38 Jorge?
00:39 OK,
00:39 so,
00:40 um,
00:40 so we have,
00:41 um,
00:42 Jorge,
00:42 not Jose,
00:43 um,
00:44 showing up,
00:45 uh,
00:46 to,
00:46 uh,
00:46 do the,
00:47 uh,
00:47 to,
00:47 to,
00:48 to kick off the discussion followed by,
00:49 uh,
00:50 uh,
00:50 by David McKenzie.
00:52 Um,
00:52 so
00:53 go ahead and I'll,
00:54 uh,
00:54 warn you around 12 or 13 minutes.
00:56 Uh,
00:57 OK,
00:57 perfect.
00:57 Thanks.
00:58 Thank you,
00:58 Art.
00:59 Uh,
00:59 just as a matter of organization,
01:00 um,
01:01 um,
01:02 let me see,
01:03 um.
01:06 Do
01:06 we have the
01:08 But what this comes from,
01:09 oh yeah,
01:09 OK,
01:10 um,
01:11 uh,
01:12 we,
01:12 the way we organize the presentation in rather than being comprehensive,
01:15 we actually selected a few topics that we think,
01:17 uh,
01:18 uh,
01:18 there's data gaps and data and knowledge gaps,
01:20 and we,
01:21 in the process of presenting these different,
01:23 different gaps,
01:24 we try to highlight the contributions of KCP over the years.
01:27 Uh,
01:28 and I would like to start with a historical perspective of,
01:30 of,
01:30 of,
01:31 of,
01:31 of the,
01:31 uh,
01:31 dimension of the private sector,
01:33 uh,
01:34 development issues
01:35 because,
01:35 uh,
01:36 the 20 years of the KCP,
01:38 uh,
01:38 match very well the development of the PhD agenda.
01:42 The
01:43 KCP started funding considerably,
01:46 uh,
01:47 considerable amounts of,
01:48 of data collection
01:49 efforts on firm level service,
01:51 uh,
01:51 which coupled with the WDR of 2005 on,
01:54 on 2005 on investment climate
01:56 and of course previous resource at the bank on.
01:59 The binding contrast to private sector growth
02:01 explain the generation of the creation of the two main
02:04 diagnostics of the private sector,
02:06 uh,
02:07 that have been,
02:08 um,
02:08 the drivers in a sense of most of
02:10 the operational policy in private sector development,
02:12 enterprise service,
02:13 and the doing business indicators,
02:15 um.
02:17 From here the link between research and
02:20 um
02:21 policy is in the cases of the in the case of the
02:23 private sector development space mostly driven through uh the creation of these
02:28 indicators that as we know are a major tool of our operational
02:31 people to to in uh in here at the bank and our clients
02:35 to galvanize all these principles of research into actual policy.
02:40 Um,
02:40 over the years,
02:41 KCP has funded
02:43 others,
02:43 the,
02:44 the research that,
02:44 uh,
02:45 substantiates other indicators like Finex,
02:47 global Financial development in,
02:48 in,
02:49 uh,
02:49 uh,
02:50 development database,
02:51 and for example,
02:52 this issue,
02:52 the,
02:53 the,
02:53 the,
02:53 the childcare development or childcare services
02:56 for the,
02:56 uh,
02:56 the.
02:57 Important childcare development services have been
02:59 demonstrated first in research and then had been made the
03:02 case for incorporation into women's terms of the law.
03:04 This is a very specific example of the
03:06 of how the I see the
03:08 the process of the linkage between research and
03:11 a policy action has taken place in in PSD.
03:14 Um,
03:15 moving on to the,
03:17 to highlight some of the issues that we wanted to highlight,
03:20 uh,
03:20 in,
03:20 in the issue of informality,
03:22 KCP has funding similar research,
03:25 uh,
03:25 on,
03:26 on,
03:26 on,
03:26 on the,
03:27 on the sector,
03:28 and,
03:28 and,
03:28 and that is,
03:29 uh,
03:30 for example,
03:30 I'm gonna draw again on the work of David
03:32 McKenzie and Christopher Guro that are the panelists today
03:35 on Sri Lanka in 2008 2010.
03:37 9
03:38 in this quite a few
03:40 papers and the research papers that have produced by them,
03:42 but in here I want to highlight the,
03:44 the fact that
03:45 uh this,
03:46 this was,
03:46 is a seminal because it,
03:47 it funded the creation of all the,
03:50 the,
03:50 the whole generation of data
03:52 from the,
03:53 from the scratch,
03:53 from starting from the full enumeration of all
03:56 the informal businesses that operated in Sri Lanka,
03:58 even in the cities of Sri Lanka where the study took place.
04:01 Um,
04:01 this is important because I want,
04:03 what,
04:03 uh,
04:03 I wanted to highlight that because really,
04:05 this really addressed
04:06 one,
04:06 the first,
04:07 uh,
04:07 data gap that I wanted to highlight on an informality,
04:09 which is the lack of census data,
04:12 of any data for that matter on informality in many countries.
04:15 Uh,
04:16 there's very limited information on the,
04:18 on the extensive margin,
04:19 the
04:20 type of the businesses that operate informally and also in the intensive margin,
04:24 uh,
04:24 businesses that operate,
04:25 uh,
04:26 um,
04:27 with informal practices or formal business operate with informal practices.
04:31 Um,
04:32 we have been working on the former,
04:34 uh,
04:34 at least in our,
04:35 in our unit,
04:36 with,
04:37 with that incorporating,
04:38 um,
04:38 new techniques,
04:40 geocoded data collection,
04:41 coupled with
04:42 a new sampling strategy
04:44 that,
04:44 uh,
04:45 takes advantage of the clustering nature of these businesses in,
04:47 in,
04:48 in,
04:48 in,
04:48 in,
04:48 in,
04:49 in,
04:49 in,
04:50 in general in the world.
04:51 And um
04:52 by um
04:54 this method,
04:54 we are able to generate representative samples of the sector in a given region.
04:59 That allow us to study whatever it was,
05:01 you know,
05:01 to do our,
05:02 our,
05:02 our,
05:03 our,
05:03 a questionnaire,
05:04 to implement our questionnaire and at the same time,
05:06 obtain an estimate
05:08 of the population totals,
05:10 in other words,
05:11 the size of the sector.
05:12 Uh,
05:13 this is something that I'm here I'm showing,
05:15 for example,
05:15 as you can see in this,
05:16 uh,
05:17 for this is from the paper
05:18 that we have written.
05:19 Uh,
05:19 we
05:20 got hold of administrative data of census data for Ezotini.
05:24 Uh,
05:24 uh,
05:24 that was fortunate because that allows us
05:26 to actually evaluate the relative merits of
05:29 this methodology vis a vis simple random sampling or certified random sampling,
05:33 and in this paper,
05:34 we show
05:34 that
05:35 there are efficiency gains in terms of the level of effort required to,
05:40 and to implement these studies
05:41 and not only that,
05:42 we can,
05:43 we're able to implement this in 3,
05:45 in between 2 and 4 months in a,
05:46 in a given region.
05:47 Um,
05:49 Moving on to more subject matters,
05:51 uh,
05:52 and to actually
05:53 highlight a few of the knowledge gaps
05:55 in informality as well
05:57 that we find,
05:57 I want to highlight three points.
05:59 The first one,
06:00 again,
06:00 and I'm,
06:00 I'm drawing from the same work of David McKenzie,
06:03 this is,
06:03 uh,
06:03 in fact,
06:04 this slide is
06:05 David's,
06:06 uh,
06:06 but in this one is the,
06:07 the lack of evidence of the benefits of formalization and the reduction of,
06:11 of cost of registration.
06:13 Uh,
06:13 this has been shown by them and many of,
06:15 uh,
06:15 several other,
06:16 other,
06:17 other studies in here.
06:18 This is again with the Sri Lanka study,
06:20 as you can see on the left.
06:21 There's a sharply decreasing demand for formalization
06:24 and this is backed out for an experiment in which,
06:27 er,
06:28 informal entrepreneurs.
06:29 were offered free registration in randomly assigned monthly profits.
06:35 And as you can see on the right-hand side,
06:38 the cumulative distribution of profits
06:40 of the control and the treatment is basically,
06:43 pretty much the same.
06:44 They look alike.
06:45 It is interesting that in this result,
06:47 that uh the lack of evidence of uh
06:50 uh
06:50 uh benefits of formalization in 2008,
06:52 2009 in Sri Lanka,
06:55 the authors mentioned that,
06:56 uh,
06:57 uh,
06:57 by interviewing these,
06:58 these entrepreneurs,
06:59 most of them expressed that they actually were aware
07:02 of the,
07:02 the process of formalization.
07:04 They just wouldn't want to know where are they taking it.
07:06 We,
07:06 I would like to compare this to our recent work in Africa,
07:09 using that methodology that I just mentioned that allow
07:11 us to do a representative sample of the sectors
07:14 in several countries.
07:15 In the,
07:15 in the context of Africa,
07:16 most of these entrepreneurs actually do respond that
07:19 they do not know anything much about formalization.
07:21 It's something that is not in the,
07:22 in the greater skin as an option.
07:25 Um,
07:26 two other points on the data and knowledge gaps that I would like to highlight is the,
07:31 the one on the left is something that has been shown by many studies already,
07:34 uh,
07:35 which is the,
07:36 uh,
07:36 the lower levels of productivity of those
07:38 informal businesses when compared to similar microfirms.
07:42 As you can see here,
07:43 these results come from a paper written so,
07:45 with the,
07:46 with the data that we have recently collected.
07:48 Uh,
07:48 and,
07:49 uh,
07:49 the informal firms are,
07:50 uh,
07:50 uh,
07:51 as you can see in,
07:52 in,
07:52 in,
07:52 in blue,
07:53 and the,
07:53 the micro firms are firms that are very much like the informal firms
07:57 in terms of sector size and even the business environment they face.
08:00 Uh,
08:01 the only differences will be fine between them
08:03 is in the management practices,
08:06 education of the entrepreneur,
08:07 and the,
08:07 uh,
08:07 uh,
08:08 experience.
08:09 Uh,
08:09 we think we still have to do a little bit more digging,
08:11 more research
08:12 to see these are the,
08:13 the real source of such differences in productivity of the rest of,
08:17 are there other things at play.
08:19 A third point as well on the,
08:22 on the knowledge gaps that I would like to highlight
08:24 in the,
08:25 in,
08:25 in the,
08:25 in,
08:26 is the effect of the presence of informality on the formal sector.
08:30 This is,
08:31 uh,
08:31 on the right-hand side,
08:32 the panel comes from a paper also produced here from Amin and others,
08:36 uh,
08:36 in which,
08:37 with more than 100 countries around the world,
08:40 and as you can see,
08:41 there's a clear difference between the firms
08:42 that face competition from formal firms,
08:45 the frames competition for informal firms
08:47 for,
08:48 and those ones that do not face competition for informal firms.
08:51 This result is a result that other
08:53 Papers,
08:54 uh,
08:54 other,
08:54 other authors are also found with all the different uh metrics,
08:58 uh,
08:59 innovation,
09:00 TFP in a different direction.
09:01 So,
09:01 I don't wanna make too much about the difference here in labor productivity,
09:05 but more of the mechanisms at play,
09:08 er,
09:08 that we need to research a little bit more because on the one hand,
09:12 uh,
09:12 one would think that,
09:13 uh,
09:13 the presence of informality is more competition for formal.
09:17 Firms
09:17 and logic would say this firms will become more effective and more innovate more,
09:22 and more productive.
09:23 Uh,
09:23 but on the other hand,
09:24 this access,
09:25 these informal firms have access,
09:27 uh,
09:27 that is in a sense,
09:28 preferential or unequal,
09:30 unfair competition,
09:31 and
09:32 therefore,
09:32 they,
09:33 they have access to,
09:34 uh,
09:35 inputs and
09:36 services,
09:36 uh,
09:37 at lower cost,
09:38 sometimes even free,
09:39 illegally.
09:41 Um,
09:42 let me just move back
09:44 quickly to the formal,
09:45 to the formal sector because I,
09:47 I don't really want to give the impression that
09:48 by all the diagnostics on the first slide and,
09:50 and the,
09:51 and the,
09:51 all the operational work that has derived from that,
09:53 that the,
09:53 in the formal sector,
09:54 there are no
09:56 data gaps.
09:56 And I think one,
09:57 in the presenting here,
09:58 one,
09:58 and I'm less optimistic that we will be doing,
10:00 we will be able to do
10:02 uh something.
10:03 effectively in the medium term,
10:05 which is the lack of administrative data
10:07 to produce research.
10:08 And I'm using here
10:09 the a KCP funded project from the Mary Howard and
10:12 others did in 2013 in which they try to collect
10:16 uh
10:16 these indicators on firm dynamics,
10:18 productivity,
10:19 and,
10:20 and,
10:20 and,
10:20 and resource allocation
10:22 uh from administrative data.
10:24 10 years later,
10:25 now 223,
10:26 it's only
10:27 a handful of countries where we have administrative data
10:29 and the issues,
10:30 uh,
10:30 the obstacles that we face in accessing,
10:32 uh,
10:33 administrative data,
10:34 uh,
10:34 are,
10:35 are relatively serious in my view because there are
10:37 legal issues on how the data was collected,
10:39 uh,
10:39 and,
10:40 and difficult to overcome.
10:41 And when one overcome those,
10:43 those issues,
10:44 these issues of comparability and quality.
10:45 So,
10:45 I think this is an issue that will remain
10:47 er for,
10:48 for some time.
10:48 And if you follow my premise here,
10:51 I think the
10:53 usefulness of the relevance of survey data becomes even more relevant,
10:56 more important.
10:57 Uh,
10:57 and so I want to highlight two points I saw as well that we,
11:00 we think there is some,
11:02 uh,
11:02 knowledge gaps and data gaps in survey data.
11:05 One is related to a point that was being discussed earlier,
11:07 is the need to have more data,
11:09 uh,
11:09 tailored for services and to understand,
11:11 well,
11:12 to,
11:12 to understand better multi-factor productivity for manufacturing as well.
11:15 But mostly for services
11:17 and the second point that I would like to highlight is the need to have more um
11:22 frequent and
11:24 uh uh frequent and uh a
11:26 larger panel datasets
11:27 that will let us address the issues,
11:29 the issues of causation and move away from the issues of correlations.
11:33 Uh,
11:34 I don't wanna over,
11:35 overstress this because,
11:37 uh,
11:37 KCP has funded quite a bit of research,
11:39 uh,
11:39 that,
11:40 uh,
11:40 on this and a lot of cross-section data
11:42 that has been very illuminating,
11:45 uh,
11:45 on the evaluating,
11:46 for example,
11:47 the efficiency of the state-provided services for the private sector.
11:50 Here,
11:50 I have one paper that I find very,
11:52 very.
11:52 Relevant,
11:53 which is by,
11:54 by Caroline Freund and Bod Rikers and others here in DC,
11:57 in which they have shown that,
11:58 uh,
11:59 and tested
12:00 that the hypothesis of greasing the wheel of bribery,
12:02 that it is,
12:03 it,
12:03 that it,
12:04 that it does not hold.
12:05 As you can see in this example here is one of the
12:07 many examples of different transactions that firms interact with the government.
12:12 Um,
12:13 let me just
12:14 finish here,
12:15 uh,
12:15 uh,
12:16 that with one final point that I would like to,
12:18 uh,
12:18 highlight,
12:18 which I think is an area in which we
12:21 will need in thinking on what I was saying
12:23 earlier on thinking on to the future,
12:26 uh,
12:26 which we,
12:27 we,
12:27 and I think this area is one because we need,
12:29 it's interesting and relevant.
12:30 Because not only is it important,
12:32 but we need to think not only on
12:33 methodologies,
12:34 how to address it,
12:35 but also
12:35 new data that we need to be collected.
12:37 And,
12:37 and this is the,
12:38 the economic cause of discrimination
12:41 along the lines of gender,
12:42 race,
12:43 and even sexual identity.
12:44 These are populations that are difficult to,
12:46 to,
12:47 to,
12:47 to collect data from,
12:49 uh,
12:49 and,
12:49 uh,
12:49 and there is not quite clear even how to go about measuring this economic cost.
12:53 That's what I'm using here.
12:55 Uh,
12:55 this very relevant paper and I find very impressive,
12:58 uh,
12:58 from in Econometrica in 2019,
13:01 which does this,
13:02 at least for the US and as you can see,
13:04 it compares the GDP growth
13:06 of the,
13:06 the actual GDP growth in blue
13:08 with the GDP growth hypothetical that will emerge from their model,
13:12 uh,
13:12 if the restrictions,
13:14 uh,
13:14 the discriminatory practices of black men and women in
13:17 the US in the 1960s will have prevailed.
13:19 I can,
13:20 you can see that it's clear in the graph,
13:21 the,
13:21 the big difference between the two,
13:23 which in the paper they say is about,
13:25 uh,
13:25 accounts for about 40% of the GDP growth uh between 1960 and 2010,
13:30 which I think is,
13:31 it makes the case for how important this is.
13:33 And with that,
13:34 They did
13:38 Great,
13:38 thanks.
13:40 So,
13:41 um,
13:42 I'm gonna dig in on sort of one topic that the KCP has helped illustrate this sort of
13:49 going back and forth between data,
13:51 research,
13:51 policy,
13:52 and,
13:52 and back to more data and more,
13:54 more research.
13:54 And so,
13:55 uh,
13:56 this is the sort of,
13:58 Question of,
13:59 you know,
13:59 why the firm size distribution looks the way it does in most developing countries.
14:03 And so,
14:04 you know,
14:04 if we look,
14:05 most firms in developing countries are very small,
14:08 many are informal,
14:09 their productivity is very low,
14:11 uh,
14:11 and so,
14:12 you know,
14:12 this raises
14:13 three questions about,
14:15 you know,
14:15 why are there so many small and unproductive firms?
14:17 How much of a problem is,
14:19 is this,
14:19 uh,
14:20 and,
14:20 and their informality,
14:21 and then,
14:21 you know,
14:22 what can policy do about it?
14:24 And so,
14:25 you know,
14:25 if we want a framework
14:27 for thinking about,
14:28 you know,
14:28 the firm's size,
14:29 then we can sort of think about
14:31 the problem of a typical firm owner who's
14:33 trying to choose their inputs of capital and labor
14:36 to maximize their profits,
14:37 and they have some sort of production function,
14:39 which has got this,
14:41 um,
14:41 capital and labor that they're buying,
14:43 and then this term theta that they're,
14:45 um,
14:46 using that,
14:46 that helps them turn their capital and labor into,
14:49 to output.
14:50 And so this suggests,
14:51 you know,
14:51 3
14:52 potential explanations for firm size,
14:54 um,
14:54 being the way it is.
14:55 So one is that firms are small because they're constrained,
14:57 and we have,
14:58 you know,
14:59 a whole other programs on finance that we're,
15:01 we're not gonna talk about today or on labor
15:03 markets that we're not gonna talk about today,
15:04 but thinking about
15:05 sort of constraints for firms and getting the inputs that they want.
15:09 The second that I'm gonna really dig into is that firms are small because they have,
15:13 uh,
15:14 a low theta.
15:14 They're just not very good at,
15:16 um,
15:16 turning their inputs into output,
15:18 and so we're gonna dig into that.
15:19 And then the,
15:20 the third point,
15:21 which is,
15:21 um,
15:22 sort of related to the informality point that Jorge was talking about is that,
15:25 you know,
15:25 firms are small because the regulations are affecting their choice of,
15:28 you know,
15:28 their production function here,
15:30 should they be formal or informal,
15:32 and,
15:32 and that can also affect the cost of their inputs.
15:35 But I'm gonna dig into this sort of middle explanation,
15:38 which is,
15:39 Um,
15:39 you know,
15:39 maybe firms are so small,
15:41 because they're just not very good at,
15:42 at managing their,
15:43 um,
15:44 process of converting inputs into outputs.
15:47 And so,
15:48 uh,
15:48 you know,
15:49 how do we go about measuring and improving
15:51 this,
15:52 this,
15:52 uh,
15:52 term here?
15:53 Well,
15:54 you know,
15:54 where were we 20 years ago?
15:56 Sort of 20 years ago,
15:57 there was this recognition that there
15:59 was something that we called managerial technology
16:02 that affected the ability of firms to translate these inputs into outputs.
16:05 It's,
16:05 but it was really a black box,
16:07 um,
16:07 so,
16:08 uh,
16:08 you know,
16:09 there's famous papers that sort of say,
16:10 whatever this is,
16:11 it's,
16:11 you know,
16:12 some firms are better at it than others,
16:13 um,
16:14 or there was this sort of view that management was really,
16:17 you know,
16:17 uh,
16:17 to do with style,
16:19 or with charisma,
16:20 or,
16:20 you know,
16:20 something that was,
16:22 was hard to
16:23 potentially teach,
16:24 and some people were just born with it,
16:25 and others,
16:26 um,
16:26 were,
16:27 were not.
16:28 Now,
16:28 you know,
16:28 of course,
16:29 there was a,
16:29 a wide range of business training programs that were based on
16:32 the idea that there were some things that you could teach,
16:34 um,
16:35 but there weren't really a lot of evaluations or no evaluations of these programs,
16:38 and there was less policy attention
16:40 to what could be done in larger firms to improve this as well.
16:44 So,
16:45 uh,
16:45 this is where the sort of first step of,
16:47 of really just trying to measure and understand,
16:50 um,
16:50 comes through.
16:51 And so,
16:52 uh,
16:52 you know,
16:52 there was a set of influential,
16:54 um,
16:56 surveys done in large firms by Nick Bloom and John Van Reenen
16:59 that,
17:00 you know,
17:00 really shaped the idea that these,
17:03 That management was,
17:04 was not just a,
17:05 a style or something that you're born with,
17:07 but it was really a set of concrete practices about how do you manage good workers
17:11 and bad workers,
17:12 how do you deal with quality,
17:13 how do you plan for the future,
17:15 things like that.
17:16 And in the KCB supported work,
17:18 uh,
17:19 Chris Woodruff,
17:19 who's,
17:19 who's gonna be one of the panelists,
17:20 and I
17:21 thought about how do we,
17:23 Take those same ideas and,
17:24 and measure this in small firms,
17:26 and do,
17:26 do these things matter in small firms?
17:28 If you walk into a lot of small firms,
17:30 they're not keeping records,
17:31 they're not doing any marketing,
17:32 they,
17:33 they don't have a budget,
17:34 does this matter?
17:35 And so what we see is,
17:36 you know,
17:37 across a range of countries,
17:38 both in the cross section over time,
17:40 it seems like,
17:42 Firms differ a lot in these practices,
17:44 that firms that do more of these practices
17:47 are more profitable,
17:48 sell more,
17:49 and um grow faster.
17:51 And so it seems like there's something about,
17:53 you know,
17:53 business practices that would be good to try and get firms to,
17:56 to improve.
17:58 Uh,
17:59 this is also something that the,
18:01 um,
18:01 enterprise surveys unit has been building on with,
18:03 with,
18:04 um,
18:04 adopting,
18:05 how do you take these,
18:06 um,
18:08 ways of measuring management that were done,
18:10 sort of,
18:11 uh,
18:12 um,
18:12 in this sort of very expensive double-blind
18:15 system and adopt them
18:17 to how they can be done in firm surveys.
18:19 And so,
18:20 you know,
18:20 they've been adopting both of these large and small firm,
18:23 um,
18:24 techniques to now make this available for a lot more countries.
18:27 So then,
18:28 you know,
18:28 once we have
18:29 this idea that,
18:30 that there's something about these practices that,
18:32 that seem to matter,
18:33 the question is,
18:34 you know,
18:35 can we do something to improve them,
18:37 and,
18:37 you know,
18:38 does this,
18:38 this matter?
18:39 And so this is where I wanna sort of
18:40 give an illustration of this interplay between,
18:43 You know,
18:43 the KCP supporting
18:44 what I call blue skies research,
18:46 where it's not really testing a policy that we think about
18:49 testing,
18:50 it's not linked closely necessarily to a
18:52 particular country operation that we have at the moment,
18:55 but it's trying to generate,
18:56 you know,
18:56 where should be things,
18:57 things be going in the future.
18:59 Um,
19:00 you know,
19:00 is,
19:00 is there even a,
19:01 should we even be trying to push policy in this direction?
19:04 And so the KCP supported this original work that,
19:08 uh,
19:08 I did
19:09 in India with,
19:10 with Nick Bloom and,
19:11 and others on
19:12 sort of a proof of concept,
19:13 which was,
19:14 if you take a badly managed firm,
19:16 can you actually go and improve management in those firms,
19:18 and would it actually matter?
19:20 And so,
19:20 you know,
19:20 this was
19:21 definitely not something that we'd recommend for policy.
19:24 We hired the most expensive consultants you could find on the planet,
19:27 probably.
19:28 Um,
19:29 uh,
19:29 you know,
19:29 very expensive global consulting firm
19:31 came in,
19:32 did
19:33 really intensive consulting on these firms,
19:36 uh,
19:36 and,
19:37 and,
19:37 you know,
19:37 spent
19:38 months in these firms
19:39 and improved management practices.
19:41 But the key thing here was that we could see,
19:43 once you improve management practices,
19:45 we could track in these data that you could get quality improvements
19:49 and see total factor productivity improve.
19:51 And so,
19:52 We could,
19:53 you know,
19:53 show not just that there was this
19:54 association between better management and better practices,
19:57 but that there was a causal link.
19:59 If you could improve management,
20:01 then
20:01 you could get this,
20:02 this increase in productivity.
20:04 So this was,
20:05 you know,
20:05 nice,
20:06 but this was not,
20:07 you know,
20:07 where
20:08 you would necessarily want to go with,
20:09 with policy,
20:11 doing the exact same thing.
20:12 And so,
20:13 you know,
20:14 then,
20:14 um,
20:15 but this is how this sort of research can then help generate
20:18 new operational ideas.
20:19 And so my operational colleagues,
20:21 you know,
20:21 are talking with finance ministers,
20:24 productivity is always mentioned as an issue.
20:26 And the question is,
20:26 you know,
20:27 what have you got that,
20:28 that tells us
20:29 how we could improve productivity?
20:30 And they said,
20:30 well,
20:31 you know,
20:31 there's this work from India that shows
20:33 that management can,
20:34 can help.
20:35 Um,
20:35 and,
20:35 and,
20:36 you know,
20:36 is there a way We could think about doing that in Colombia.
20:38 And so then I worked with them in,
20:40 in,
20:40 in the government in Colombia,
20:41 and we sort of tested,
20:43 could you do this,
20:43 but could you do this in a more scalable,
20:46 more cost-effective way?
20:47 And we tested a group-based consulting,
20:49 uh,
20:50 method,
20:50 as well as this individual consulting method.
20:52 And you see in this graph,
20:53 you know,
20:54 across a whole range of management practices,
20:56 you get improvements,
20:57 um,
20:57 that are very similar from this group-based consulting approach than you.
21:01 from,
21:01 from the individual approach at a third of the cost.
21:03 So,
21:04 you know,
21:04 we get this,
21:04 this method that can improve,
21:07 um,
21:07 some growth in Colombia,
21:09 can be done at a lower cost,
21:10 and,
21:11 you know,
21:11 then that link,
21:12 links to,
21:13 sort of,
21:13 the next round of policy where now,
21:15 now that that's been done in Colombia,
21:17 countries are coming and saying,
21:19 oh yeah,
21:19 let's see if we can do this in Ghana,
21:21 we can do this in Malawi,
21:22 etc.
21:23 And so,
21:24 um,
21:25 You know,
21:26 that's,
21:26 that's sort of,
21:27 you know,
21:27 from that,
21:28 that blue skies research leading to new government projects.
21:32 And then,
21:32 you know,
21:32 one of the things that comes up is that there's just not many of these firms.
21:36 We're trying to do this a lot with sort of small samples.
21:38 And so,
21:38 one of the things that the KCP has also been,
21:41 um,
21:41 helping us do is more methodological improvements on,
21:44 you know,
21:44 how do we incorporate prior information,
21:46 how do we make the most of these small samples.
21:48 And so some of the
21:49 most recent work we've been doing on
21:51 uh new Bayesian impact evaluation methods has been supported by the KCP there.
21:56 So then,
21:57 you know,
21:57 that's large firms.
21:58 What about small firms?
21:59 Well,
22:00 you know,
22:00 some of,
22:00 uh,
22:01 one of our panelists,
22:02 Dean Kalin was,
22:03 you know,
22:03 really one of the first,
22:04 I think the first to,
22:05 to look at,
22:06 um,
22:07 was this business training effective.
22:09 And there's been sort of many studies since that have found that,
22:12 you know,
22:12 some of these business training programs
22:14 can help
22:14 firms,
22:15 even the very smallest firms,
22:17 improve their business practices,
22:18 improve profits and sales.
22:20 But,
22:20 um,
22:21 you know,
22:21 it's,
22:21 it's
22:22 somewhat mixed evidence.
22:24 Uh,
22:24 in some cases,
22:25 it's not always as
22:26 factors as,
22:27 as we'd like.
22:28 And it seems quite hard to scale some of these in-person classroom-based,
22:32 um,
22:32 training.
22:33 And so,
22:33 that's where,
22:34 you know,
22:35 I think two of the other directions that have been
22:36 coming through some of the most recent KCP supported work,
22:39 I've been doing,
22:39 my colleagues,
22:40 Miriam and,
22:40 and Kayle here have been working in Brazil to see if
22:43 you could sort of start with a much lighter touch approach,
22:46 give some firms diagnostics,
22:48 um,
22:49 use this to identify,
22:50 you know,
22:50 which firms you might then want to support in the second
22:53 stage and get firms to demand some of the training themselves.
22:56 Um,
22:57 unfortunately,
22:58 our,
22:58 our panelist,
22:59 um,
22:59 Geo,
23:00 who,
23:00 who's,
23:00 uh,
23:01 on a flight right now,
23:02 uh,
23:03 she heads
23:03 this,
23:04 uh,
23:05 organization of,
23:05 uh,
23:06 female entrepreneur,
23:07 um,
23:08 training.
23:08 Programs in,
23:09 in Mexico called CREEA,
23:11 and,
23:11 you know,
23:11 we've been working with them to see if you could do,
23:13 use digital technology to try and scale um training and,
23:17 and,
23:17 you know,
23:17 provide it remotely,
23:18 and we've been working,
23:19 you know,
23:19 both in Mexico and Guatemala,
23:21 you know,
23:21 using Zoom,
23:22 use,
23:23 using what we're doing right now to do training to,
23:25 to people all across the country.
23:27 So,
23:27 you know,
23:27 we're,
23:28 we're trying to learn that.
23:29 So,
23:29 just,
23:30 to sort of,
23:30 you know,
23:30 where,
23:31 where do we go next?
23:32 I guess,
23:32 you know,
23:33 there's a couple of recurring themes of persistent puzzles.
23:35 And so,
23:36 one is just,
23:37 you know,
23:37 firms are just very different from one another.
23:39 Management capacity varies dramatically across,
23:42 um,
23:42 and within countries.
23:44 There's just lots of variation in the scope for growth.
23:46 And so the question is,
23:47 you know,
23:47 how can we identify which firms we should be targeting,
23:50 um,
23:50 policies to,
23:51 um,
23:52 more?
23:52 How can we scale this in a way that,
23:55 um,
23:56 You know,
23:56 the market works better,
23:58 how much does the government need to be involved
23:59 versus just trying to get the market to work,
24:02 um,
24:02 better,
24:02 and then we need to,
24:03 you know,
24:03 work on collecting better,
24:04 um,
24:05 data on firm,
24:06 firm outcomes,
24:07 which is quite tricky.
24:08 And so,
24:09 you know,
24:09 as a,
24:10 uh,
24:10 you know,
24:10 then there's these new areas of private sector development that
24:13 are really coming to the fore in terms of,
24:15 you know,
24:15 how do we collect more information on,
24:17 on green growth,
24:18 green PSD,
24:20 um,
24:20 transactional government,
24:21 the enterprise surveys,
24:22 you know,
24:22 have been starting to measure these practices.
24:25 And,
24:25 you know,
24:25 hopefully,
24:25 this is the next phase of,
24:27 you know,
24:27 we collect some data,
24:28 we identify some diagnostics,
24:30 and now we can,
24:31 you know,
24:31 start some interventions based on,
24:33 on that.
24:33 And so,
24:34 um,
24:35 you,
24:35 you know,
24:35 there's,
24:36 there's just sort of a lot of scope now with big data,
24:38 with AI with other approaches to try and improve our,
24:41 our survey technology,
24:42 and I hope this is sort of gonna be,
24:44 you know,
24:44 we start a whole new batch of,
24:46 uh,
24:46 diagnostics that can help get this cycle going again.
24:49 So let me stop there.
24:53 Thanks very much,
24:54 Jorge and David,
24:55 um,
24:56 uh,
24:56 for great presentations,
24:57 uh,
24:58 or presentation,
24:59 I guess very integrated,
25:00 um,
25:01 so we'll turn now to our panelists and as they've already mentioned,
25:04 uh,
25:05 unfortunately,
25:06 uh,
25:06 Giovanna Hernandez Constantino,
25:08 um,
25:09 who was supposed to be our first panelist is
25:11 unable to,
25:11 to join us because of,
25:13 uh,
25:13 travel mishaps,
25:14 um.
25:15 Uh,
25:15 but we do have,
25:16 um,
25:17 uh,
25:17 3 other excellent panelists who we'll go to in turn in the same format as before.
25:22 So,
25:22 uh,
25:22 we'll begin with Dean Carlin,
25:24 who,
25:24 uh,
25:25 needs a little introduction,
25:26 um,
25:26 and,
25:27 uh,
25:27 currently is the chief economist at USAID.
25:30 Um,
25:31 then we'll turn to Deborah Revoltella,
25:33 who is the chief economist and director of
25:35 the Economics Department at the European Investment Bank.
25:38 Um,
25:38 and finally,
25:39 we'll turn to,
25:40 uh,
25:40 Chris Woodruff,
25:41 who is at Oxford and,
25:42 uh,
25:43 uh,
25:43 also I think needs a little introduction given the,
25:46 uh,
25:46 the importance of the work that he has,
25:48 uh,
25:48 uh,
25:50 he has done and that we frequently see cited here in the bank.
25:53 Um,
25:54 so,
25:54 starting with you,
25:55 Dean,
25:56 uh,
25:56 we'll,
25:56 uh,
25:57 we'll go with,
25:57 uh,
25:58 my,
25:58 sorry,
25:59 I think the commitment was to 8 minutes each.
26:01 So,
26:01 if you can sort of aim for that and,
26:03 uh,
26:04 Uh,
26:05 and,
26:06 and then we'll have a chance for a discussion.
26:08 Uh,
26:08 so
26:09 over to you,
26:09 Dean.
26:10 OK,
26:10 hold on,
26:10 I'm starting a timer.
26:11 Don't start the 8 minutes yet.
26:14 There we go.
26:15 Um,
26:15 hi,
26:15 everyone.
26:16 I,
26:16 I wish I was over there.
26:17 Um,
26:18 sorry,
26:18 I,
26:18 I ended up too tight,
26:20 um,
26:21 on the,
26:22 on the,
26:22 on the walkover.
26:23 So,
26:24 I'm going,
26:24 I'm doing the lazy path.
26:26 Um,
26:27 so,
26:27 I wanted to share 33 thoughts that,
26:31 um,
26:32 that
26:33 also feed,
26:34 you know,
26:35 build on a lot of the things that we just discussed,
26:37 um,
26:37 and,
26:38 and,
26:39 and are by no means in lieu of,
26:40 I think there's a lot of,
26:42 like,
26:42 really important questions that,
26:44 um,
26:45 that,
26:46 that we just heard about that are being posed.
26:48 There's a lot of different directions where more research is needed to understand.
26:52 You know,
26:52 things like the technology question that David was just talking about,
26:56 um,
26:56 for,
26:57 for achieving scale and,
26:58 um,
26:59 and,
26:59 you know,
26:59 we,
27:00 we have an overall
27:02 I don't want to say conundrum,
27:03 but a little bit of a conundrum of the fact that,
27:06 you know,
27:06 I think,
27:07 I think
27:08 we can kind of go backwards and say,
27:10 if someone asks you for a point estimate,
27:13 You can find the study that,
27:14 that finds it when it comes to things that are
27:16 in the space of promoting managerial capital or business training.
27:20 And,
27:20 um,
27:21 and that shouldn't be seen as a,
27:23 as a,
27:24 in some,
27:24 in some respects,
27:25 I think that should be seen as a motivation,
27:27 not as a,
27:28 not as a deterrent,
27:29 a motivation for,
27:30 for the kind of work that we've just heard about,
27:32 that's really trying to get in more granular understanding of
27:36 Of both,
27:37 how do we do,
27:38 uh,
27:39 diagnostics as well as how do we,
27:41 how do we
27:42 separate out and understand what types of implementation strategies are,
27:46 are,
27:46 are working and the intersection there and hopefully we'll start,
27:49 um,
27:49 getting more and more insights.
27:50 But as,
27:51 as you heard,
27:51 there's,
27:52 you know,
27:52 there's definitely
27:53 You know,
27:54 we,
27:54 we,
27:54 you know,
27:55 the state of the evidence is definitely better than um
27:57 we can,
27:58 that everything goes,
27:59 right?
27:59 It's,
27:59 it,
28:00 we,
28:00 we do,
28:00 there are some clear patterns that are coming through that,
28:03 that show
28:04 um
28:05 policymakers who are not interested in research,
28:07 you just want to use evidence that does show
28:09 we,
28:09 you know,
28:10 we do have important insights as you've just been hearing about.
28:13 Um,
28:14 so I want to highlight three areas that I see as kind of big gaps that in this,
28:19 in this area that,
28:20 and when I say big gaps,
28:21 I want to be clear that like,
28:23 there's definitely like really good,
28:24 smart,
28:25 um,
28:26 careful,
28:26 um,
28:27 policymakers into,
28:28 you know,
28:29 implementers and researchers working on these,
28:31 so,
28:31 you know,
28:32 To a lot of people in the room,
28:33 um,
28:34 there's a
28:35 none of these three are gonna be like,
28:36 wow,
28:36 we never thought of that before.
28:37 But,
28:38 um,
28:38 but I do see these as three areas that are
28:40 continue to be,
28:41 um,
28:43 areas of big,
28:44 big gaps relative to
28:46 the kind of standard,
28:47 um,
28:48 impact evaluation.
28:50 The first is very much motivated by my baldness.
28:53 Um,
28:55 And um,
28:56 although the jokes don't seem to work too well over Zoom,
28:58 but um I have a real gripe with hair,
29:01 um,
29:02 and,
29:02 and with hair salons.
29:05 And it's just to make a very simple point about general equilibrium effects
29:09 that
29:10 we don't,
29:10 you know,
29:11 we talk about them a lot.
29:12 We obviously thought about them a lot.
29:13 There's been a lot of good research on it and,
29:15 you know,
29:16 by,
29:16 by people,
29:17 by David and others,
29:18 and,
29:19 um,
29:19 and,
29:20 but,
29:20 you know,
29:21 obviously,
29:21 no pun intended,
29:22 there's not gonna be a very simple generalized answer to generalized equilibrium,
29:26 general equilibrium effects.
29:28 Um,
29:29 and yet we desperately need to understand more about them.
29:31 Um,
29:32 there's been,
29:33 you know,
29:33 a lot of,
29:34 when we look at the expansion of microenterprise in the US,
29:36 I was always struck by
29:38 how often the prototypical enterprise that,
29:41 that was being promoted as a microenterprise in the United States was hair salons.
29:45 And this is why I make the hair joke,
29:47 because like,
29:48 adding more hair salons in,
29:50 you know,
29:50 in most communities,
29:52 uh,
29:52 you know,
29:52 there's only a fixed amount of hair in the world.
29:55 And,
29:55 um,
29:56 as much as I might resent those people,
29:58 um,
29:58 they're adding more hair salons does not lead to more haircuts.
30:02 Um,
30:03 and so,
30:04 uh,
30:04 you know,
30:05 what,
30:05 what's really happening here,
30:06 even if you set up a nice randomized trial,
30:08 you're very rarely gonna fit,
30:09 you know,
30:09 kind of detect that kind of effect.
30:11 It's,
30:11 you need a very large study,
30:13 um,
30:14 you need a lot of the stars to be aligned.
30:16 But we do need more work to understand this.
30:19 Um,
30:19 and we need to first separate out and not,
30:22 and not do what I just did and talk about general,
30:25 general equilibrium effects as some sort of monolithic thing and,
30:27 and
30:28 separate it out into information effects,
30:31 um,
30:31 and price effects and then kind of industrial organization,
30:34 competition
30:36 structure effects.
30:37 And
30:38 the,
30:38 so that's the first thing we need to do when we think about G and then,
30:41 you know,
30:41 could also think about kind of macro multiplier kind of um effects as well.
30:45 So,
30:46 you know,
30:47 we need to separate out what we mean by this.
30:49 We need to do,
30:49 we need to try to incorporate when we can better,
30:52 better disentangling of these effects.
30:54 And then we,
30:55 you know,
30:56 we need to,
30:56 we need to kind of rope in our macro friends as,
30:59 as micro people and um to,
31:01 to um integrate
31:03 some of the,
31:04 the micro studies into better macro modeling of these kinds of issues to understand
31:08 more about
31:09 how to set policy.
31:11 That um can hopefully use GE effects to improve impacts rather than,
31:16 um,
31:17 rather than,
31:17 you know,
31:18 the opposite,
31:18 which is the fear in many cases that they're going to,
31:21 um,
31:21 diminish treatment effects and,
31:23 and all we're doing is business stealing by promoting some businesses.
31:27 Um,
31:27 the second
31:28 gap that I,
31:30 that I,
31:30 um,
31:31 have
31:32 been,
31:32 um,
31:32 kind of obsessed with for a while,
31:34 and I think it's actually much less,
31:36 much less research compared to like the general equilibrium is,
31:39 um,
31:39 a long time ago I wrote a,
31:40 uh,
31:40 a kind of a thought piece with Sendel Mono,
31:42 and been to Anno called Three Anomalies.
31:46 Um,
31:46 and one of those anomalies was,
31:48 why don't we see more people share
31:50 businesses?
31:52 Two-person,
31:52 three-person equity firms,
31:54 um,
31:54 as the on-ramp.
31:56 And,
31:57 and it's really easy to think through examples and
32:00 household enterprises where there's a clear economy of scale.
32:03 Um,
32:04 by having or economy of scope,
32:06 depending on the business style,
32:08 goat herding,
32:09 piggeries,
32:10 cow sheds,
32:11 where a cow shed is a fixed cost and having multiple households keep their cows.
32:14 There's lots of examples like this that we,
32:17 that are easy to tell.
32:18 And,
32:19 you know,
32:19 what's the obstacle here in understanding more about those,
32:23 um,
32:23 those challenges?
32:24 Is it moral hazard?
32:25 Is it training?
32:26 Is it,
32:27 um,
32:27 is it trust?
32:28 Is it just,
32:29 um,
32:30 is it,
32:30 is it just wrong and the returns aren't that high?
32:32 To some of these,
32:33 but it does seem like on paper,
32:35 some of these returns could be really,
32:37 um,
32:38 really remarkable.
32:39 And,
32:40 and we need to see more work on that.
32:41 So,
32:41 that's not quite on the,
32:42 that's,
32:43 I would put that in the small,
32:44 but not the medium size.
32:45 This is about the,
32:46 that,
32:46 that bridge from getting from micro to small firms.
32:50 And the third
32:52 is
32:53 why,
32:54 here's a puzzle that I've always been struck by is,
32:57 given that we do see
32:59 some
33:00 um really strong impacts from,
33:02 you often more tailored and customized
33:05 managerial capital intervention.
33:07 I'm thinking about things like
33:09 the India study that David did,
33:11 I'm thinking about the Mexico study that I
33:13 did with Antoinette um Shore and Miriam Brune.
33:16 Um,
33:17 and,
33:18 um,
33:19 these showed really high returns.
33:21 Um,
33:22 we saw some pretty impressive improvement in,
33:25 in labor force,
33:26 in returns to,
33:27 um,
33:28 in kind of total factor productivity in,
33:29 in,
33:30 in Mexico.
33:32 So how is it possible,
33:34 and David's heard me ask this question before,
33:36 so,
33:36 how is it possible
33:38 that these consulting firms can be so good.
33:41 At giving out advice,
33:43 and yet they're not.
33:45 Like expanding
33:47 amazingly well and getting really,
33:49 you know,
33:49 rich themselves.
33:50 What,
33:50 what's the,
33:51 what's the problem in the consulting market,
33:54 if this really is such a,
33:56 a clear path to improving SME,
33:59 um,
34:00 total,
34:00 you know,
34:00 profits?
34:01 Where's the issue?
34:02 Is it risk?
34:03 Is there heterogeneity and quality,
34:05 and people don't know how to choose firms?
34:08 Is it information?
34:10 Um,
34:10 is it,
34:11 um,
34:11 is it capital?
34:13 Is it just a capital constraints or a contracting issue?
34:15 And that's my 8 minutes.
34:17 Um,
34:18 you know,
34:18 where's,
34:19 what,
34:19 where's the market failure rate in the market for consulting?
34:22 Like,
34:22 and,
34:23 and I,
34:23 I'm,
34:24 I'm still,
34:25 I,
34:25 I,
34:25 I was struck by this from the study that we did in Mexico.
34:28 I've been struck by this by any,
34:30 any study that shows big success for
34:32 managerial capital interventions from a for-profit entity.
34:36 That is not then expanding magnificently themselves.
34:39 Like what's,
34:40 what's the,
34:41 what's the constraints in that market?
34:44 So I'll close there.
34:47 Thanks very much,
34:48 Dean.
34:49 Um,
34:49 so you want to talk to more macroeconomists,
34:51 I'll just remind you that macroeconomists mean something
34:53 very different when they talk about haircuts.
34:56 Uh,
34:56 so just
34:57 bear that in mind when you start those discussions.
35:00 Um,
35:00 all right,
35:01 over to Deborah.
35:05 Thank,
35:06 thank you very much and thank you very much for inviting me for uh for uh this
35:11 uh panel and uh this uh discussion and I'm also very sorry not to be with you today
35:16 um in Washington but uh being only remotely,
35:20 but for me it would have been a,
35:21 a longer traveling coming there.
35:25 So,
35:25 um,
35:26 I,
35:27 I read the task,
35:29 uh,
35:29 for being in this panel in terms of,
35:31 uh,
35:31 suggesting topics that I think,
35:34 uh,
35:34 in terms of private sector development at this point in time,
35:38 we should,
35:38 uh,
35:39 uh,
35:39 really put in the agenda and also,
35:42 uh,
35:42 think where we need to invest.
35:45 Much more uh on uh data and uh research and I think uh
35:50 um where I'm coming from and uh probably is also for uh the institution I work with
35:57 um one of the uh key concern uh in this moment that uh
36:02 that we are looking at on the research side is trying to understand.
36:07 Um,
36:08 what motivate,
36:09 uh,
36:10 the transition,
36:11 uh,
36:11 the,
36:11 the,
36:11 the transformation of of firms of,
36:14 uh,
36:15 of companies
36:16 in the net zero transition and,
36:19 uh,
36:20 addressing climate change.
36:22 And I think on that point.
36:23 Point of view,
36:24 we really,
36:26 we really have an issue in terms of data availability and understanding.
36:31 We're investing much more in terms of data and the survey also can play a rule
36:37 would be quite relevant.
36:40 Um,
36:41 where I'm coming from,
36:42 uh,
36:43 um,
36:44 we,
36:45 uh,
36:45 we know that,
36:46 uh,
36:46 uh,
36:47 that the transformation will,
36:49 uh,
36:49 will the net zero transition will lead to a complete change in relative prices,
36:55 reallocation of resources,
36:57 and economic power
36:59 within and across sectors,
37:02 and the,
37:02 the incentive.
37:03 to move as a company and to transform really
37:07 change a lot depending on where you position it.
37:11 If you hold
37:13 assets that will become stranded assets,
37:17 if you are in a good position for developing a new technologies or not,
37:23 whether you have access to.
37:26 Resources that are necessary for
37:29 the green transition
37:31 and also what is the policy framework
37:35 around you that incentivize or disincentivize the transition.
37:42 So
37:43 to me what
37:45 what would be a very useful.
37:49 Processes to,
37:51 to start looking more at
37:53 uh what motivates firms to invest in the net zero transition,
37:59 to transform and radically transform
38:02 why some firms
38:03 are lagging behind and others are.
38:07 Accelerating the transition while some
38:11 play into
38:12 gaining the upside while others just
38:15 play into
38:18 amortizing the current assets
38:22 before doing anything in terms of transformation.
38:25 And the complexity of doing the work is really that there are
38:30 so many elements at hands both on the regulatory point of side,
38:34 on the firm specific point of file side,
38:37 the counter-specific
38:40 point that uh that actually you,
38:42 you really need to,
38:44 to
38:44 go very granular in terms of
38:47 um
38:48 of uh uh of data and uh.
38:51 And information and that's why I was thinking that
38:55 working with a survey may be a useful compliment.
39:00 And maybe if I still have
39:02 one second after this long introduction to explain why I'm showing this
39:09 to you,
39:09 I will try to share a presentation where I will just
39:14 show a couple of slides,
39:18 um.
39:20 And I think,
39:20 uh,
39:20 can you see it?
39:22 Yes,
39:22 it looks good.
39:24 OK,
39:25 and I think I have to,
39:28 OK.
39:29 Um,
39:30 what I will show you something that,
39:32 uh,
39:32 we launched,
39:33 uh,
39:34 7 years ago at the European Investment Bank is a
39:36 survey of more than 12,000 firms at the European level,
39:41 representative for single countries and with a benchmark at the European,
39:46 uh,
39:46 at the US level
39:48 and also is the links to balance sheet data of the firms.
39:52 So we combine the balance sheet information of the firms.
39:56 With uh
39:57 qualitative and quantitative data that uh we
40:01 um
40:02 we uh gain from uh from uh the survey
40:06 exercise
40:07 and what we can do is,
40:09 uh,
40:09 uh,
40:10 first of all ask the firms to think about the barrier
40:13 on investment here in particular in the NASA survey we had a
40:18 very strong rule of energy costs as a
40:20 barrier to investment and uncertainty at the European.
40:24 Level
40:24 and that's allowed us also combined to the other
40:29 variables to understand what was happening in terms of
40:33 climate investment and energy efficiency investment in Europe.
40:37 So what we find out is exactly that
40:41 in the moment in which in Europe we had the
40:44 energy crisis last year coming after the war in Ukraine,
40:48 what we had was that
40:51 Energy costs were a driver for
40:55 investment in energy efficiency,
40:57 but on the other side,
41:00 the
41:02 uncertainty was
41:03 was a drag to investment overall.
41:06 But when we look at energy efficiency,
41:08 the net effect
41:10 of
41:11 the push for the energy cost and the uncertainty
41:15 left a small positive.
41:17 So increasing the probability.
41:19 Ability for firms
41:20 to invest in energy efficiency and in fact we see it at the European level with
41:27 very strong investment in energy efficiency this
41:29 year in the private sector coming up.
41:32 What happened with the climate action investment that is a wider concept
41:37 not only energy efficiency but also adaptation and all other element of mitigation
41:43 was in fact that the
41:45 um
41:47 Positive impact of the energy cost was a
41:50 completely balanced by the fact of uncertainty.
41:54 And we explain it also because on a policy
41:56 point of view there were various instruments that were
42:00 um
42:01 Used also to contain energy prices to
42:05 um
42:06 to,
42:06 to support the firms during the crisis
42:10 that uh distorted incentive on investment.
42:13 So
42:14 what you had is that the combination of
42:16 uncertainty and energy costs left a positive stimulus
42:20 for energy efficiency but it didn't change the
42:23 probability of firms to invest in climate action.
42:26 And that's something important uh to understand also on a policy point of view,
42:31 to know which kind of instrument and policies that work
42:35 to stimulate the climate investment action.
42:39 And then uh uh an example of what we are doing in the survey,
42:43 we are asking
42:44 firms what they do for mitigation and for adaptation
42:49 and also their perception,
42:50 of course,
42:51 uh,
42:51 before I don't show it of physical risk and transition risk.
42:55 What you see on mitigation,
42:57 we ask them which kind of measure they implement.
43:01 You see a lot of use of energy efficiency
43:04 investment of waste minimization that helps us understanding.
43:08 The regulation of matters
43:10 and actually in Europe we see the effect of a regulation
43:13 in imposing a waste minimization for firms and that's why that's
43:17 an area so developed.
43:20 But it's also interesting that the um
43:23 the investment in new less polluting technologies or activities which is
43:28 the transformation of firms moving
43:31 either changing technologies or moving in
43:34 business areas and business activities.
43:37 That are less polluting.
43:39 What we see there and it's the current work that we are developing,
43:43 we are trying to look at the most energy intensive sectors
43:47 and looking at what motivates the firms that are actually
43:52 investing to transform and those that are
43:55 not transforming
43:56 and with more binding regulation coming in,
44:00 you would expect that those that are non-transforming,
44:04 they are trying to use the assets.
44:06 In the end,
44:06 and then they will just close the activities
44:09 while uh the others uh start transforming the business already now.
44:14 I think uh
44:15 further analysis and further work in this direction is very
44:19 important because of the structural shift that the economy will have
44:24 and understanding how to,
44:26 um,
44:28 how the reallocation of resources and also
44:31 the transformation of firms that will happen is extremely important.
44:35 With that,
44:36 uh,
44:36 I think,
44:36 uh,
44:37 uh,
44:37 I had in the presentation a couple of additional slides to show a project that
44:42 we developed actually also with uh Jorge
44:45 and with DBRD based on the enterprise survey
44:49 both on the North African region and on the East and North Africa second project.
44:57 There as well we were looking at the physical climate risk,
45:01 green investment and trying to understand that the
45:04 uh drivers for firm transformation.
45:07 And again I think that was an early
45:10 um attempt with an early version of a green model for the enterprise survey.
45:15 I think that going uh um.
45:19 The more direct
45:21 as possible,
45:22 as direct as possible with the new set of definition that now
45:27 start becoming uh um standard and understood
45:30 also by the corporate sector in terms of
45:33 what is the physical risk,
45:35 what is the transitional risk,
45:36 and what can be done in a mitigation and adaptation.
45:39 Having a more spelled out question I would be.
45:43 Useful and would help also uh around the world
45:46 a better understanding of what motivates the film.
45:49 I hope this uh um
45:51 helps in the discussion and uh
45:55 and uh that I didn't bring uh topics completely out
45:59 of uh of uh what you wanted to discuss,
46:02 but I think that the climate part is very important to consider.
46:10 Thanks very much,
46:11 Deborah.
46:12 Um,
46:12 uh,
46:13 Chris,
46:13 here,
46:14 uh,
46:14 our last panelist.
46:15 Go ahead.
46:16 Great.
46:16 Thank you.
46:17 And,
46:17 and thanks for,
46:18 uh,
46:18 the chance to,
46:19 to,
46:19 to,
46:20 to speak to you today.
46:21 Um,
46:21 first,
46:22 I wanna say congratulations,
46:23 uh,
46:23 to,
46:24 to KCP for,
46:25 for the work you've done over the years and,
46:27 and the amount of time.
46:28 I think
46:29 I wanna sort of both reflect back a bit over that period of time and what we've,
46:33 I think,
46:34 accomplished and also kind of think about
46:36 what
46:36 really is
46:38 is maybe where we've done less,
46:40 where we've been,
46:40 where we've made less progress,
46:42 and where I think the next 20 years,
46:44 uh,
46:44 might,
46:44 uh,
46:45 uh,
46:45 might,
46:45 might focus.
46:46 And,
46:46 and as I sort of reflected back at,
46:49 at,
46:49 at,
46:50 on the 20 years,
46:50 I,
46:51 I thought about the work that David and I started doing,
46:54 you know,
46:54 I guess a bit more than 20 years ago.
46:57 Um,
46:57 and thinking about
47:00 Um,
47:00 data
47:01 and microenterprises and where we were with kind of how we understood
47:06 how,
47:06 what profits were,
47:07 how firms,
47:09 um,
47:09 thought about
47:10 investments,
47:11 thought about returns and things,
47:13 uh,
47:13 things of that sort in places where
47:15 people,
47:16 uh,
47:16 didn't even,
47:17 uh,
47:17 keep,
47:17 uh,
47:18 any,
47:18 any written records.
47:20 Um,
47:20 and the literature was really quite
47:22 sparse in a sense.
47:24 Um.
47:24 I think
47:25 one of the,
47:26 if,
47:26 if I,
47:27 I think about that time,
47:28 that one of the surprising
47:29 um pieces of data that was out there was the,
47:33 the,
47:33 uh,
47:34 the Enemin survey in,
47:35 in Mexico,
47:36 the microenterprise survey in Mexico,
47:37 which,
47:37 which David and I used early,
47:39 early on,
47:40 um,
47:41 which was in some senses,
47:42 way,
47:42 way ahead of its time.
47:43 And if I think about the 20 years and where we've come from,
47:47 from that point in time,
47:48 um,
47:49 From the perspective of data
47:51 and how we understand these firms,
47:53 it's really quite amazing,
47:55 uh,
47:56 amazing progress.
47:57 And,
47:58 uh,
47:58 And,
47:59 uh,
47:59 as,
47:59 as was pointed out,
48:00 a lot of this work,
48:01 you know,
48:02 uh,
48:02 important work was funded by,
48:03 uh,
48:04 you know,
48:04 by the bank and by,
48:06 by KCP,
48:07 um,
48:07 but it,
48:08 but we're,
48:09 we're at a point where it's,
48:10 we're way beyond kind of,
48:12 uh,
48:13 measuring,
48:14 um,
48:15 uh,
48:16 what,
48:16 what firm,
48:16 how firms keep,
48:18 how,
48:18 how firms think about profits and,
48:20 and,
48:20 and,
48:21 and,
48:21 uh,
48:21 and other things,
48:22 and thinking about
48:23 really using technology and using experiments and using,
48:27 um,
48:28 Uh,
48:28 other kinds of,
48:29 uh,
48:30 information to think about,
48:31 um.
48:32 Uh,
48:33 effort at,
48:34 at,
48:34 at,
48:35 at their,
48:35 when,
48:35 when they're working in the enterprise and other aspects of,
48:39 of,
48:39 of productivity,
48:41 um,
48:42 that is,
48:43 uh,
48:43 is,
48:44 is,
48:44 you know,
48:44 really quite some,
48:46 some startling progress.
48:47 And I,
48:47 and I think the progress has been
48:49 most,
48:50 uh,
48:51 uh,
48:51 impressive and,
48:52 and strongest in the area of,
48:54 of microenterprises and thinking about
48:57 these very small scale,
48:58 uh,
48:58 these very,
48:59 very small scale,
49:00 uh,
49:00 enterprises.
49:01 I think recently,
49:02 there's also been a fair amount of work in very large enterprises.
49:06 And,
49:07 um,
49:08 and in,
49:08 in a way,
49:09 these are
49:10 the,
49:10 you know,
49:11 the,
49:11 what's,
49:12 what's missing is,
49:13 is the missing middle.
49:14 And it's not just cause the firms,
49:16 there are not as many firms there,
49:17 but because we,
49:18 we haven't,
49:19 I think,
49:20 focused as,
49:21 with some notable exceptions,
49:22 haven't focused as much attention on,
49:25 on sort of mid-sized firms.
49:26 And
49:27 It's kind of obvious as researchers why that's the case,
49:30 that we end up,
49:31 uh,
49:31 going to the field and seeing
49:33 massive numbers of very small-scale firms.
49:36 It's easy to get a sample,
49:37 we can do,
49:38 we can do work,
49:39 or we can go develop a relationship with a large-scale firm,
49:42 and the firm's large enough that we can do
49:45 experiments and other work,
49:46 uh,
49:47 within a single or a handful of,
49:49 of,
49:49 of very large firms.
49:51 Um,
49:51 but if I think about the kinds of firms that are,
49:55 um,
49:55 participating in
49:57 incubators,
49:57 accelerators,
49:58 the sort of fast-growth,
49:59 you know,
50:00 sort of,
50:00 uh,
50:01 uh,
50:02 uh,
50:03 mid-sized firms,
50:05 um,
50:05 there's,
50:06 I think,
50:07 quite a lot,
50:08 you know,
50:08 less work in that area.
50:10 And I think that's a place where,
50:12 um,
50:13 where
50:13 we need,
50:14 uh,
50:14 we,
50:15 you know,
50:15 we need,
50:15 uh,
50:16 we need more attention.
50:18 And I wanna,
50:19 I,
50:19 I guess I wanna say,
50:20 I think,
50:21 um,
50:21 and,
50:22 and
50:23 Jorge,
50:23 you,
50:23 you mentioned the difficulty with,
50:25 with administrative data and the lack of administrative data,
50:28 and I think
50:29 that's right.
50:30 There's often in many countries,
50:32 a lack of administrative data,
50:33 but we're beginning to see
50:35 now
50:36 in,
50:36 in some quite,
50:37 you know,
50:38 in some quite important
50:39 instances of people getting access to
50:43 value-added transaction tax data,
50:46 to trade data,
50:47 to uh corporate income tax data,
50:49 uh,
50:49 and being able to sort of merge across data sets
50:53 uh in India,
50:54 and Pakistan,
50:55 and Uganda,
50:55 and Kenya,
50:56 and other,
50:57 in other countries.
50:58 And
50:59 those
51:00 Data,
51:01 I think,
51:01 have
51:02 uh,
51:02 a tremendous amount of promise
51:05 in
51:06 helping us understand
51:07 um
51:08 both firm dynamics generally
51:10 and the sorts of general equilibrium effects that,
51:12 that,
51:13 uh,
51:13 that,
51:13 that Dean,
51:14 that Dean was,
51:14 was talking about.
51:16 Um,
51:16 they have the advantage of being comprehensive.
51:18 They have the advantage of being,
51:20 um,
51:21 Uh,
51:21 you know,
51:22 available over a long period of time.
51:24 Um,
51:25 of course,
51:25 only for formal,
51:26 for formal firms.
51:28 But,
51:28 uh,
51:29 but the,
51:30 these,
51:30 you know,
51:31 these sort of faster growing mid-sized firms
51:33 are,
51:33 are likely,
51:34 uh,
51:34 to be.
51:34 And so,
51:35 I think the
51:37 The
51:38 one of the challenges we face as,
51:40 as researchers and,
51:42 and,
51:42 and,
51:43 and when,
51:43 when working with,
51:44 with policymakers is
51:46 figuring out ways
51:47 to,
51:48 to make those,
51:49 uh,
51:49 those kinds of data more accessible.
51:52 And,
51:52 uh,
51:53 for example,
51:53 Uganda is setting up a secure data center.
51:56 South Africa has,
51:57 has done some work in setting up secure data
51:59 centers to make the data more broadly available.
52:02 I mean,
52:03 I think over
52:04 The past,
52:05 they've been
52:06 typically available when you get the right relationship and you some,
52:09 and you,
52:10 and you get access to the,
52:11 to the data.
52:12 But,
52:12 uh,
52:13 but I think
52:14 the challenge of making those more systematically
52:17 available is one that I think the bank can,
52:19 uh,
52:20 can,
52:20 can play a,
52:21 can play a role in,
52:21 and I think would have,
52:22 uh,
52:23 uh,
52:24 if you build it,
52:24 they will,
52:25 they will come kind of sense of generating a lot of research,
52:28 uh,
52:28 uh,
52:29 a lot of research in this,
52:30 um,
52:31 in,
52:31 in this area.
52:32 And
52:34 Uh,
52:34 one of the things that I would say,
52:36 I again see as an area that's developing now,
52:40 and
52:41 an area that's likely to grow is exactly on the sort of,
52:44 uh,
52:45 um,
52:46 interaction between,
52:48 uh,
52:49 microeconomists and macroeconomists,
52:51 um,
52:52 or at least
52:52 the interest in macroeconomists of,
52:55 uh,
52:55 in,
52:55 in the private sector and in firms.
52:57 Um.
52:58 I run a,
52:59 uh,
53:00 some of you will know,
53:01 I run a program
53:02 funded by uh FCDO,
53:05 uh,
53:05 uh,
53:06 called Private Enterprise Development in Low-income Countries,
53:08 a grants program.
53:09 There's a sister program called Structural Transition,
53:12 uh,
53:12 uh,
53:13 and Economic Growth,
53:14 uh,
53:15 which is basically a macro program.
53:17 And SEG funds
53:18 really quite a substantial amount of
53:21 work on firms and work on the private sector,
53:23 and much of it is exactly in this area of kind of thinking about,
53:27 um,
53:27 Uh,
53:28 you know,
53:29 how we think about general equilibrium effects,
53:31 how we think about using,
53:33 uh,
53:33 structural models,
53:35 um,
53:35 uh,
53:36 to,
53:36 uh,
53:37 to,
53:37 to,
53:38 to,
53:38 to come up with sort of policy counterfactuals.
53:41 We're not quite there yet in,
53:42 in marrying the,
53:43 the microeconomics with the,
53:45 with the macroeconomics,
53:46 but,
53:47 but there have been
53:48 Really important moves in the,
53:49 in that direction in the,
53:50 in the last,
53:51 just,
53:52 you know,
53:52 recent
53:53 years.
53:53 And I think that's also a place where,
53:56 um,
53:56 where there's a lot of potential,
53:58 uh,
53:58 in the,
53:59 in,
53:59 in the next,
54:00 uh,
54:01 in the next,
54:01 in the next 20 years.
54:02 So,
54:03 I'll,
54:03 I'll stop,
54:04 I'll leave it at that and,
54:05 and say,
54:06 I think,
54:06 you know,
54:08 Measurement data,
54:09 uh,
54:10 and David mentioned the management practices.
54:12 Another area where,
54:14 um,
54:15 uh,
54:15 you know,
54:16 uh,
54:16 as,
54:16 as the saying in management goes,
54:18 what gets measured gets managed.
54:20 You know,
54:20 when,
54:21 when we're able to measure things within,
54:23 within,
54:23 uh,
54:24 Uh,
54:25 within these firms and across these firms,
54:27 um,
54:28 you know,
54:28 we were able to,
54:30 to make much more progress in,
54:32 in thinking about what policies are and what,
54:34 what interventions are that,
54:35 that will,
54:36 uh,
54:36 that will,
54:37 will benefit
54:38 firms and,
54:38 and,
54:39 and firm growth.
54:39 So,
54:41 Data measurement,
54:42 yes,
54:43 the way we've been doing it with surveys and with,
54:45 and with developing kind of new,
54:47 um,
54:48 relatively small sample,
54:50 uh,
54:50 rich data sets.
54:52 But I think also data,
54:54 uh,
54:54 thinking more about,
54:55 uh,
54:56 the administrative data and the,
54:57 and the,
54:58 the ability to kind of
55:00 Reach much larger populations of firms,
55:02 uh,
55:03 where
55:04 firms that are less common when we go to the field will,
55:07 will,
55:07 will be,
55:08 uh,
55:08 available and,
55:09 and,
55:10 uh,
55:10 and there in,
55:11 in,
55:11 in larger numbers,
55:12 uh,
55:13 uh,
55:13 will be,
55:14 uh,
55:14 will be important as we go,
55:15 as we go forward.
55:16 And I'll,
55:17 I'll leave it there.
55:23 Thanks very much,
55:24 Chris,
55:24 and uh I'm not sure if you're able to connect to the first part of the session,
55:27 but you're really kind of bringing us full circle because
55:29 we had a lot of discussion about the importance of,
55:32 of drawing on administrative data,
55:34 um,
55:35 you know,
55:35 both on firms,
55:36 on tax records,
55:37 and so on as a way of,
55:38 uh,
55:39 of generating insights and also the,
55:41 the difficulties in accessing this data.
55:43 So we have,
55:44 excuse me,
55:45 we have about 20 minutes um remaining uh for questions.
55:49 Um,
55:50 so the floor is open,
55:51 we'll start with some in the room,
55:52 and if we have any online,
55:53 please,
55:53 please let me know,
55:54 Karina.
55:55 So,
55:55 uh,
55:55 Sergio and then Daria,
55:57 and then Karina if we have some people online.
56:00 For us.
56:01 And,
56:01 uh,
56:01 thanks for the presentation.
56:02 It's very interesting.
56:03 Um,
56:04 one thing for advertising and,
56:06 um,
56:06 following what Chris mentioned is at the macro group and also using KCP funds.
56:10 We are trying to
56:11 link the micro and the macro
56:13 aspects of the economy using,
56:15 um,
56:16 more widely available data at the country level or cross country.
56:20 But,
56:20 uh,
56:21 my question was,
56:21 uh,
56:22 basically to David and,
56:24 and Dean,
56:25 you mentioned,
56:26 uh,
56:26 this issue of a black box of what entrepreneurship,
56:29 uh,
56:30 abilities are.
56:31 Um,
56:31 and you mentioned the value of,
56:33 uh,
56:33 doing consulting and training,
56:35 uh,
56:35 to these entrepreneurs that can increase performance over time.
56:40 But it's still,
56:41 for the audience,
56:42 that's a black box.
56:44 So,
56:44 I was wondering whether you can uh provide a bit more examples
56:48 of what are the practices that you use to train these entrepreneurs
56:53 and what are the most effective uh training practices that you have uh seen
56:57 that have an effect on,
56:58 on,
56:59 on entrepreneurial activity later on.
57:05 Um,
57:06 yes,
57:06 my question,
57:07 um,
57:07 I think really to whoever wants to pick it up,
57:10 but,
57:11 uh,
57:11 so there is,
57:12 there's been a lot of discussion about what happens within the firm,
57:15 but from the trade side,
57:17 we see that really this firm to firm linkages are very important.
57:20 So I was wondering whether there is,
57:22 uh,
57:23 um,
57:23 you know,
57:24 a program there to perhaps bridge that dimension.
57:30 Now,
57:31 I wanted to,
57:31 I mean,
57:31 it's maybe more common than the question,
57:33 but um I think,
57:34 you know,
57:35 the,
57:36 the link between blue sky type of,
57:38 of research and then policy impact as,
57:40 as uh David really,
57:41 uh,
57:42 you know,
57:42 outlined very well and uh documented well,
57:45 is really,
57:45 you know,
57:45 I think when you look at David's presentation,
57:47 he really schemes
57:49 uh through all the important steps in between getting that first,
57:53 you know,
57:54 uh,
57:54 paper that was academically received.
57:56 But then making those linkages with policymakers and really,
58:00 you know,
58:00 it's not like you have a paper in the
58:02 top five journal and suddenly people hear your messages
58:05 in the circles of influence,
58:06 right?
58:06 So,
58:07 I think kind of
58:08 thinking about together about the,
58:09 the importance of KCP financing and the work of
58:11 the bank more generally in making these linkages and,
58:14 and can we
58:16 think a little bit differently about KCP financing and making
58:18 this work a little bit easier for researchers to really
58:21 uh make that link.
58:25 So,
58:25 there's a question from Alvaro Gonzalez,
58:28 which is actually similar to what Sergio just asked,
58:30 and how can we focus more on
58:33 the firm,
58:33 the kind of firm dynamics that can promote better uh managed firms,
58:38 and
58:38 how do we better identify factors that really improve the quality of management.
58:42 This is more for David.
58:44 Thanks.
58:47 Anybody else in the room wanna come in?
58:49 Norman,
58:49 please.
58:54 Thank you.
58:55 Uh,
58:55 great presentation,
58:57 David,
58:57 Jorge.
58:58 Uh,
58:59 I want to
59:00 make a comment on,
59:01 on administrative data,
59:03 uh,
59:04 that has been the subject of the,
59:05 of this session and the previous one.
59:08 And,
59:08 uh,
59:09 present maybe a counterpoint
59:11 on,
59:11 uh,
59:12 on the use of administrative data.
59:14 I,
59:15 I do wonder in some,
59:17 for some countries,
59:18 whether this data
59:20 are truly reliable.
59:23 Um,
59:23 and whether we should be concerned about
59:26 that reliability.
59:27 Uh,
59:28 and if that is a concern,
59:30 then how can we
59:32 Combine
59:34 administrative data with
59:36 independently collected data
59:38 like we do in the,
59:39 the Enterprise survey program
59:41 to uh improve the
59:45 reliability of such data.
59:47 And the second point that is related
59:50 is,
59:50 uh,
59:51 well,
59:51 some countries are making this data
59:54 available,
59:55 but then,
59:56 can we extrapolate
59:57 to other countries
59:58 that don't make the data available
1:00:01 for some reason.
1:00:03 Um,
1:00:04 and
1:00:05 my conjecture is that
1:00:07 extrapolation is very hard.
1:00:09 And that's why we also need independent data collection.
1:00:12 So,
1:00:13 2 issues of administrative data.
1:00:15 Thank you.
1:00:18 Oh,
1:00:18 thank you very much,
1:00:18 Norman.
1:00:19 I was actually going to raise uh perhaps even a sharper version of that question,
1:00:22 because it's not just a question of reliability,
1:00:24 it's also a question of,
1:00:26 in some cases,
1:00:27 political influence,
1:00:28 agendas,
1:00:28 and so on,
1:00:29 uh,
1:00:30 that we have to find ways to filter through with administrative data,
1:00:33 and I,
1:00:34 I think,
1:00:34 well,
1:00:34 particularly in your be Ready project,
1:00:37 you're really trying to navigate that,
1:00:38 um.
1:00:39 Uh,
1:00:40 let,
1:00:40 let me add,
1:00:41 uh,
1:00:42 two other quick questions to the mix,
1:00:44 and then we'll go back to the,
1:00:45 uh,
1:00:45 the speakers and the panelists for,
1:00:47 you know,
1:00:48 1 or 2 minutes each for,
1:00:49 uh,
1:00:49 for responses.
1:00:51 Um,
1:00:52 uh,
1:00:52 I mean,
1:00:52 I'm,
1:00:52 I'm picking up a little bit on
1:00:54 sort of the,
1:00:54 the similar question that,
1:00:56 that Dean posed.
1:00:57 Um,
1:00:58 uh,
1:00:58 so Dean asked the question about,
1:01:00 you know,
1:01:00 why is it that consulting firms aren't getting rich?
1:01:02 And I think the flip side of that question is,
1:01:05 You know,
1:01:06 why isn't management sort of like
1:01:08 just another input like capital and labor,
1:01:10 and why are the sort of puzzles different in,
1:01:13 in that case?
1:01:14 Like,
1:01:14 you know,
1:01:15 for example,
1:01:15 one could take the view that firms don't invest in management because
1:01:19 they're small,
1:01:19 and one might even make a case that there's some kind of a,
1:01:22 you know,
1:01:22 size trap mechanism that's driving that.
1:01:25 Although I know from a paper that you and I wrote together,
1:01:27 David,
1:01:27 that we're not very sympathetic to that view.
1:01:29 So,
1:01:29 if it's not that trap view,
1:01:31 uh,
1:01:31 what would the other view be?
1:01:33 Um,
1:01:33 and I have a question for,
1:01:34 for Deborah as well.
1:01:35 So you showed some very interesting stuff on the green side,
1:01:38 and I think you made a very compelling case that
1:01:40 thinking about green incentives for firms are important.
1:01:43 One question I had seeing the slides from your EIBIS
1:01:47 survey is,
1:01:49 you know,
1:01:49 what do we learn from these questions about
1:01:52 sort of
1:01:53 private incentives versus external benefits of
1:01:57 investments that firms are making,
1:01:59 right?
1:01:59 Because the trite answer always when
1:02:01 we sort of confront that question and say,
1:02:03 Well,
1:02:03 your governments should just put carbon taxes and,
1:02:05 right?
1:02:06 But,
1:02:06 but we know that governments can't put carbon taxes.
1:02:08 So,
1:02:08 we really need to understand,
1:02:10 well,
1:02:10 in the absence of carbon taxes,
1:02:12 what are,
1:02:13 what are the things that are actually in firms' private interest to do
1:02:16 and what are the things that
1:02:17 they're doing insufficiently because of externalities,
1:02:20 and how can these surveys tell us more about that?
1:02:23 So,
1:02:24 with that,
1:02:24 let's go first to Jorge and David,
1:02:27 um,
1:02:28 and then we'll go to the panelists,
1:02:29 and if I could ask you to be a bit disciplined about sticking to,
1:02:32 you know,
1:02:32 2 minutes each,
1:02:33 we'll be able to wrap up on time.
1:02:36 Yes,
1:02:36 a few of the topics perhaps,
1:02:37 uh,
1:02:38 addressing first the,
1:02:38 the issue of the um initiative data
1:02:42 uh for uh Christopher,
1:02:43 I think he
1:02:44 raised that and Norman's point.
1:02:46 Uh,
1:02:46 the,
1:02:47 the,
1:02:47 the,
1:02:47 I think I agree with Christopher in the sense that,
1:02:49 uh,
1:02:49 yes,
1:02:49 you do,
1:02:50 we do have a big gain access to some very good data in some specific countries.
1:02:55 The point I was making,
1:02:56 uh,
1:02:56 is actually to the point he said that word systematic.
1:02:58 Around the globe.
1:02:59 It's,
1:03:00 I don't think this is a very promising.
1:03:01 It's,
1:03:01 it's kind of like Nietzsche,
1:03:02 what we can find in some certain countries,
1:03:04 great,
1:03:05 but when we try to think of that around the world,
1:03:07 it's very limited.
1:03:09 Uh,
1:03:09 and then there's restrictions.
1:03:10 I am interested in that among the countries that you mentioned,
1:03:13 one of those is one of the countries in which I have my faced most difficulties
1:03:17 accessing any data for,
1:03:18 for any kind of work,
1:03:19 for research
1:03:20 and,
1:03:20 and,
1:03:21 and,
1:03:21 and.
1:03:21 And even for the implementary services in the case,
1:03:23 the case of South Africa.
1:03:24 Um,
1:03:25 and I do know that we gain access to that access just to use as an example,
1:03:28 we have gained access to that,
1:03:29 but a very restrictive way of using the data and the bank has,
1:03:33 has profited from that.
1:03:34 But the point I was making is that it's a little bit
1:03:36 uh difficult to put your money into that,
1:03:38 into,
1:03:39 into,
1:03:39 uh,
1:03:39 on a global basis and I realized we,
1:03:42 the World Bank probably should play a role on that.
1:03:44 Uh,
1:03:44 uh,
1:03:45 and,
1:03:45 uh,
1:03:45 it,
1:03:45 it,
1:03:46 it has come to me several times,
1:03:47 the request of me playing the role in that way
1:03:49 is because I interact with a lot of these,
1:03:50 uh,
1:03:50 with all the countries around the world in
1:03:52 my role as manager of interpersonallysis unit,
1:03:54 and at least a little bit of my frustration that I was expressing is,
1:03:57 uh,
1:03:57 my experience from,
1:03:58 from building from all those interactions in which is,
1:04:00 uh,
1:04:00 is,
1:04:01 is,
1:04:01 is,
1:04:01 it's,
1:04:01 it's not,
1:04:02 it,
1:04:02 it,
1:04:02 you just,
1:04:03 we just,
1:04:03 um,
1:04:04 bump into issues of legality and it's not nothing researchy,
1:04:07 it's poli,
1:04:07 it's purely political,
1:04:08 the access to this administrative data.
1:04:10 Um.
1:04:12 I think,
1:04:12 uh,
1:04:12 on,
1:04:12 on,
1:04:13 on,
1:04:13 on,
1:04:13 I just wanted to touch base on,
1:04:14 on,
1:04:14 on Deborah's point as well on the green practices in the green of firms
1:04:18 in,
1:04:19 uh,
1:04:19 in the,
1:04:20 the,
1:04:20 the,
1:04:20 she made the case of
1:04:22 the work we did together in the past
1:04:23 and,
1:04:23 uh,
1:04:24 uh,
1:04:24 but she was making the case of uh trying to build more into the service,
1:04:28 uh,
1:04:28 this transition of the motivation of the firms into the transition.
1:04:31 I fully agree and I would be happy to explore
1:04:33 how they're doing it and see how we explore,
1:04:35 but I also So,
1:04:36 I would add,
1:04:36 moving to the table that I would like to discuss how,
1:04:38 what is the best way to,
1:04:39 way to do that?
1:04:40 Is it survey data,
1:04:41 the best way to address that or is there other
1:04:43 methods of data collection that probably would be better.
1:04:45 I'm not very keen on asking direct questions about incentives
1:04:49 to firms because I'm more keen on observing how they,
1:04:51 how they act
1:04:52 in trying to do that because
1:04:54 um this,
1:04:55 those direct questions on,
1:04:57 on or perceptions in a sense,
1:04:58 uh can be misleading.
1:05:00 Um,
1:05:00 I think just a little bit on,
1:05:02 on,
1:05:02 I think,
1:05:02 uh,
1:05:02 on the,
1:05:03 on the,
1:05:03 within the firm.
1:05:04 There's a little bit of work on,
1:05:05 on my,
1:05:05 on my team,
1:05:06 more on the research side,
1:05:06 not on data collection within the firm on that,
1:05:08 on what's happening within the firm,
1:05:10 uh,
1:05:10 but I,
1:05:11 but it's true.
1:05:11 It's not,
1:05:11 not much systematic in the,
1:05:13 at least on the data collection effort that we do have.
1:05:15 It's a little more on the,
1:05:16 on the,
1:05:17 on the research side of,
1:05:18 of a couple of people in my team.
1:05:22 So,
1:05:23 um,
1:05:23 yeah,
1:05:24 thanks,
1:05:24 thanks to all these great,
1:05:25 um,
1:05:26 comments and,
1:05:27 and discussing comments.
1:05:28 Uh,
1:05:29 so let me just pick up on a couple of things,
1:05:30 and so,
1:05:31 one is this,
1:05:32 um,
1:05:33 point we sort of get all the time,
1:05:34 which is,
1:05:36 We,
1:05:36 you know,
1:05:36 we work for years,
1:05:38 we,
1:05:38 we
1:05:39 do a big sample,
1:05:40 we,
1:05:40 we painstakingly look at this data,
1:05:42 and if we,
1:05:43 and,
1:05:43 you know,
1:05:44 a lot of the times we find things that don't work,
1:05:45 and then when we find something that works,
1:05:47 The question is always,
1:05:48 well,
1:05:48 if it works,
1:05:49 why isn't the private sector already doing this already,
1:05:51 and,
1:05:51 you know,
1:05:52 I had to,
1:05:53 you know,
1:05:53 interview
1:05:54 3000 firms,
1:05:55 trace them over,
1:05:57 you know,
1:05:57 3 years,
1:05:58 um,
1:05:58 do a lot of econometrics,
1:05:59 and I can barely detect the impact,
1:06:01 and then we think that everybody should know that this is,
1:06:04 you know,
1:06:04 what,
1:06:05 what's going on.
1:06:05 And so,
1:06:06 um,
1:06:07 you know,
1:06:07 I think
1:06:08 it,
1:06:09 you know,
1:06:09 a lot of this stuff,
1:06:10 it's,
1:06:10 it's actually,
1:06:11 even when you've gone through this,
1:06:13 there's just so much other stuff that hits firms that it's actually very hard.
1:06:16 Hard for them to know whether,
1:06:18 um,
1:06:18 what they're doing works or,
1:06:20 or not.
1:06:20 And so,
1:06:20 this is,
1:06:21 I think,
1:06:21 one of the big failures in the consulting market is,
1:06:23 is that
1:06:24 it's,
1:06:25 um,
1:06:25 you know,
1:06:25 it's an experience,
1:06:26 good.
1:06:26 You don't know what you're getting before you go in,
1:06:28 as,
1:06:29 as Sergio said,
1:06:30 you know,
1:06:30 you told me about consulting,
1:06:31 but I don't really know what that,
1:06:33 that is.
1:06:33 Um,
1:06:34 you know,
1:06:34 there's a sort of set of practices,
1:06:36 like,
1:06:36 you know,
1:06:37 even at a general level,
1:06:38 if I say,
1:06:38 well,
1:06:38 it's about quality improvement and logistics improvement and HR practices,
1:06:41 well,
1:06:41 that's great,
1:06:42 but then what specific practices,
1:06:44 and,
1:06:44 you know,
1:06:44 that's all very context-specific.
1:06:46 Um,
1:06:46 so,
1:06:47 I think it's,
1:06:47 you know,
1:06:48 very hard for firms to know what they're getting ex ante.
1:06:50 We do see that when firms have gone through this,
1:06:52 they're more likely to go back and buy a bit of
1:06:55 this consulting on the market themselves.
1:06:56 So I think there is this sort of experience,
1:06:58 good
1:06:59 aspect.
1:06:59 But I think it's just really hard,
1:07:01 um,
1:07:01 you know,
1:07:01 there's this great paper I like on advertising.
1:07:04 Where,
1:07:05 um,
1:07:05 firms that are doing experiments with 2 million customers
1:07:08 can't tell whether they're getting 50% return or 0% return
1:07:12 on that advertising expenditure.
1:07:13 It's just like super hard to know when so much other stuff hits your firm,
1:07:17 whether anything you do,
1:07:19 do works or not.
1:07:20 And so,
1:07:20 I think this is like,
1:07:21 you know,
1:07:22 one of the big challenges for us.
1:07:23 And so that's where,
1:07:23 you know,
1:07:24 the,
1:07:24 the sort of macro factors of,
1:07:26 um,
1:07:27 You know,
1:07:27 what's driving firm dynamics and,
1:07:30 and,
1:07:30 you know,
1:07:30 what else can we have at that sort of ecosystem level in terms of competition policy,
1:07:34 and some of our
1:07:35 colleagues have been working on that,
1:07:37 and,
1:07:37 and,
1:07:38 uh,
1:07:38 um,
1:07:39 you know,
1:07:39 what can
1:07:40 large firms do through their value chains of trying to help
1:07:44 their,
1:07:44 their customers improve management and,
1:07:46 you know,
1:07:46 we see trade and having this role in improving.
1:07:48 Um,
1:07:49 and upgrading as well.
1:07:50 And so I think it's,
1:07:50 you know,
1:07:51 trying to look from the bottom up and,
1:07:52 and the top down,
1:07:53 but I think the sort of point of,
1:07:54 you know,
1:07:54 when you finally find something that maybe works,
1:07:57 um,
1:07:57 why has everyone not done it already?
1:07:58 Uh,
1:07:59 and yet,
1:07:59 you know,
1:08:00 I,
1:08:00 I do a lot,
1:08:01 we also do lots of well-intended things that don't work,
1:08:03 um,
1:08:04 is,
1:08:04 is there.
1:08:04 So let me stop there and I'll,
1:08:06 um,
1:08:06 you know,
1:08:06 I'm sure there's other questions,
1:08:07 but I wanna give the panelists time to respond.
1:08:10 OK,
1:08:11 thanks.
1:08:11 Let's go to the panelist's kind of in reverse order.
1:08:13 So,
1:08:14 um,
1:08:15 uh,
1:08:15 Chris,
1:08:15 do you have a couple of observations?
1:08:18 Yeah,
1:08:19 sure.
1:08:19 So,
1:08:19 so a couple of things.
1:08:20 Uh,
1:08:21 one is,
1:08:21 one is on the question of,
1:08:23 um,
1:08:23 on what works in training and,
1:08:25 and,
1:08:25 and so forth.
1:08:26 I,
1:08:26 I wanted to bring it back to,
1:08:27 to one of the slides that,
1:08:28 that Jorge had about,
1:08:30 um,
1:08:30 uh,
1:08:31 about gender and discrimination and so forth and say that
1:08:33 I think that,
1:08:34 um,
1:08:36 There's one issue is what you can do to sort
1:08:38 of improve entrepreneurial ability among
1:08:40 people who are currently entrepreneurs,
1:08:41 but I also think we shouldn't lose sight of the fact that
1:08:44 we,
1:08:44 we,
1:08:44 and a lot of these economies,
1:08:45 we're not doing a very good job of
1:08:47 selecting who becomes an entrepreneur to begin with.
1:08:49 And we need to think hard about that,
1:08:51 about that,
1:08:52 uh,
1:08:52 that,
1:08:52 uh,
1:08:53 question as,
1:08:53 as,
1:08:54 as,
1:08:54 as well.
1:08:55 And,
1:08:55 and,
1:08:55 um,
1:08:57 You know,
1:08:57 we get to better entrepreneurs,
1:08:59 uh,
1:08:59 either from,
1:09:00 from better selection or from,
1:09:01 or from,
1:09:02 or from better training.
1:09:03 And then,
1:09:04 on administrative data,
1:09:05 so,
1:09:05 so funny,
1:09:06 I've spent most of my career kind of
1:09:08 justifying why survey data should be thought
1:09:10 of as as reliable as administrative data.
1:09:12 So it's,
1:09:13 so I,
1:09:13 I,
1:09:13 you know,
1:09:13 I'm,
1:09:14 I'm happy to hear,
1:09:15 uh,
1:09:15 people say that,
1:09:16 well,
1:09:16 maybe,
1:09:16 and,
1:09:16 and it,
1:09:17 and it's clearly gonna be the case when we think about
1:09:19 taxes and,
1:09:20 and,
1:09:20 and,
1:09:20 you know,
1:09:21 reasons that people would,
1:09:22 would,
1:09:23 would skew administrative data.
1:09:24 I,
1:09:25 I think I think that the data are valuable,
1:09:27 um,
1:09:27 and I think the research that's been,
1:09:29 that's been done when thinking about
1:09:31 firm to firm networks and how networks,
1:09:33 uh,
1:09:34 that,
1:09:34 that,
1:09:34 that,
1:09:34 the data that,
1:09:35 that,
1:09:36 um,
1:09:36 that,
1:09:36 that,
1:09:37 that the VAT data in particular allow us to,
1:09:40 um,
1:09:40 the,
1:09:41 the patterns of people that,
1:09:42 that those data allow us to understand is,
1:09:44 is,
1:09:44 is,
1:09:45 uh,
1:09:45 I'm gonna say unique cause it's really difficult.
1:09:48 It would be really difficult to,
1:09:49 to,
1:09:49 to,
1:09:50 uh,
1:09:50 uh,
1:09:51 generate any kind of broader,
1:09:53 um,
1:09:54 Uh,
1:09:54 survey.
1:09:55 Um,
1:09:56 and,
1:09:56 uh,
1:09:57 in,
1:09:57 in a broader sense,
1:09:58 in a,
1:09:58 in,
1:09:58 in a survey.
1:10:00 And,
1:10:00 um,
1:10:01 and then the question of,
1:10:03 of
1:10:04 representation of the countries where these data are available.
1:10:08 If I go back 10 years,
1:10:09 I'm,
1:10:09 I don't think VAT data were available in any country.
1:10:12 Now,
1:10:12 we've got,
1:10:13 you know,
1:10:13 4 or 5 countries in Africa where,
1:10:16 um,
1:10:17 uh,
1:10:17 you know,
1:10:17 where there are at least 3 countries where they're very broadly available and,
1:10:21 and,
1:10:21 and,
1:10:21 uh,
1:10:22 And Ethiopia as well,
1:10:24 where people are using them.
1:10:25 So,
1:10:26 I,
1:10:26 I think
1:10:27 that
1:10:28 it's likely that
1:10:30 perhaps by demonstrating that other countries will,
1:10:32 will be,
1:10:33 will begin to,
1:10:34 will,
1:10:34 will begin to see these,
1:10:35 these,
1:10:36 these,
1:10:36 these kinds of data pop up in other places.
1:10:38 So,
1:10:39 um,
1:10:39 I,
1:10:39 I,
1:10:40 I,
1:10:40 I
1:10:40 Take these,
1:10:41 I take these points,
1:10:42 but I,
1:10:42 but I think that um
1:10:44 there are a set of questions that I think it's gonna be quite challenging for us to,
1:10:49 to address with,
1:10:50 uh,
1:10:51 uh,
1:10:51 with,
1:10:52 uh,
1:10:52 with,
1:10:52 with survey data where I see uh promise in those,
1:10:55 uh,
1:10:56 in,
1:10:56 in those,
1:10:56 in those data.
1:10:57 And
1:10:58 thanks.
1:11:01 Thanks,
1:11:01 Chris.
1:11:02 Deborah.
1:11:04 On my,
1:11:05 on my side,
1:11:06 maybe,
1:11:06 um,
1:11:07 one of the things that I was uh trying to pass
1:11:10 as a message is also the fact that that combining data,
1:11:14 data,
1:11:14 uh,
1:11:15 it's,
1:11:15 uh,
1:11:16 extremely valuable.
1:11:17 On our side,
1:11:18 what we do is we combine the balance sheet information of the firms that are.
1:11:25 Uh,
1:11:25 externally,
1:11:26 so we have hard data combined to survey data
1:11:29 and then matching with,
1:11:31 uh,
1:11:31 all the data,
1:11:33 data sources,
1:11:34 databases on the location of the firms,
1:11:37 and,
1:11:37 uh,
1:11:38 all information that you have on the
1:11:40 overall external environment and I think it's uh
1:11:43 the combination of information.
1:11:45 So that brings a lot of value in understanding because then you can
1:11:51 understand how much is the firm behavior,
1:11:54 how much is the external environment,
1:11:56 how much is the policy intervention
1:11:59 that influence what the sing,
1:12:03 what motivates the action of the firm.
1:12:06 So I think that particularly in topics
1:12:10 extremely difficult to analyze like the the the green transition
1:12:15 where
1:12:15 data,
1:12:16 data are difficult,
1:12:17 the combination of
1:12:19 different
1:12:21 ways of seeing the same phenomenon,
1:12:24 including the survey data is important.
1:12:31 Thanks,
1:12:31 Deborah.
1:12:32 Uh,
1:12:32 so,
1:12:33 Dean,
1:12:33 over to you for the last word.
1:12:38 Sorry,
1:12:39 sorry.
1:12:40 Um,
1:12:41 uh,
1:12:41 yeah,
1:12:41 no,
1:12:41 I think this is,
1:12:43 you know,
1:12:43 for what it's worth,
1:12:44 I think this discussion on data has been fascinating,
1:12:46 and
1:12:47 I,
1:12:47 I,
1:12:48 um,
1:12:49 haven't worked as much with it personally,
1:12:51 with the kind of administrative data,
1:12:52 but I think it,
1:12:53 it hits on some of the,
1:12:54 I,
1:12:55 I,
1:12:55 I'm,
1:12:55 I'm excited by the,
1:12:56 by the promise of the integration,
1:12:58 both because of the scale,
1:12:59 administrative data possibilities and
1:13:02 And also because I think of the importance in trying to get at some of the,
1:13:06 the general equilibrium issues that I,
1:13:07 that I was talking about,
1:13:08 um,
1:13:09 that,
1:13:09 that is,
1:13:10 you know,
1:13:10 clearly a,
1:13:11 a path to trying to,
1:13:13 um,
1:13:13 to try to tackle some of those.
1:13:14 So it was,
1:13:15 uh,
1:13:16 thanks for having me on this and I was,
1:13:18 you know,
1:13:18 really was,
1:13:19 it was great listening to everybody and
1:13:21 learning more about what everybody's um up to and where
1:13:24 everybody is thinking there's big gaps and rooms for progress.
1:13:27 So I'm excited for the next 20 years.
1:13:33 Thanks very much,
1:13:34 Dean.
1:13:34 That's actually sort of a segue into what I wanted to say by
1:13:38 um
1:13:38 to,
1:13:39 to wrap up this.
1:13:39 So this is not just wrapping up this session in today's event,
1:13:42 but also the series of 3 events that we've put on to,
1:13:45 to celebrate these 20 years of KCP.
1:13:48 I keep focusing on the 20 years,
1:13:49 but I occasionally drop in and $80 million of over
1:13:53 $80 million of donor funded research uh during this time.
1:13:56 So,
1:13:56 this has really been a very significant effort on the part of our KCP donors,
1:14:01 which I think over the years we've had over
1:14:03 20 different donors to KCP who have contributed,
1:14:07 uh,
1:14:07 to make a lot of this research possible and to advance on,
1:14:10 on,
1:14:10 on this agenda.
1:14:12 I did want to
1:14:13 particularly acknowledge that our current donors to the,
1:14:16 the current round of KCP,
1:14:17 so,
1:14:18 uh,
1:14:18 CETA and Sweden.
1:14:19 Uh,
1:14:20 um,
1:14:20 France,
1:14:21 uh,
1:14:21 the government of Japan,
1:14:22 and also the EU through two different,
1:14:24 uh,
1:14:25 uh,
1:14:25 DGs that have been
1:14:26 contributing to,
1:14:27 uh,
1:14:28 to KCB funded research.
1:14:29 Uh,
1:14:30 um,
1:14:31 I think this is,
1:14:31 uh,
1:14:32 it's a great foundation for the next 20 years and
1:14:34 we're looking forward to being able to do much more.
1:14:37 Um,
1:14:37 just to
1:14:38 close up,
1:14:39 I also did want to give a big thank you to
1:14:41 Karina and Bintao for all the work that they've done in,
1:14:43 uh,
1:14:44 in putting together again,
1:14:45 not just today's event,
1:14:46 but the 3 events and for making KCP run.
1:14:49 Also,
1:14:50 this is now becoming more of an inside the bank comment for a second,
1:14:53 but
1:14:53 I did also want to,
1:14:54 uh,
1:14:55 thank Bintao in particular,
1:14:57 who's been with KCP for as long as I can remember.
1:15:00 Although my memory is failing these days,
1:15:02 so,
1:15:02 you know,
1:15:02 maybe that isn't such a big compliment,
1:15:04 but Bintao is going to be moving on shortly to,
1:15:07 uh,
1:15:07 do a new opportunity in DFI,
1:15:09 right?
1:15:10 No.
1:15:11 Water,
1:15:12 sorry,
1:15:12 in the water global practice
1:15:14 and sitting beside Bintao and just to introduce
1:15:16 quickly and we'll do it more systematically later,
1:15:18 but we have Anna Bokina who is joining us from the Africa region,
1:15:21 who will be stepping into Bintao's role.
1:15:23 So,
1:15:24 um,
1:15:25 welcome,
1:15:25 uh,
1:15:25 welcome Anna as well.
1:15:27 So with that,
1:15:28 um,
1:15:29 uh,
1:15:30 uh,
1:15:30 let me again thank our speakers,
1:15:32 David,
1:15:32 Jorge,
1:15:33 and our,
1:15:33 our panelists,
1:15:34 Deborah,
1:15:35 Dean,
1:15:35 and,
1:15:36 uh,
1:15:36 and Chris,
1:15:37 and,
1:15:37 uh,
1:15:37 you know,
1:15:38 big round of applause and everybody for all of their,
1:15:41 uh,
1:15:41 their hard work to make this happen and for all the interesting insights.
1:15:44 Thank you very much.
- add-style
- lp-body-content