Evidence to Fix Bureaucracies: The Sequel

videoType

dynamic-media

videoDmUrl

keyFrameImage

timestamp

00:01 Well,

00:01 welcome everybody.

00:02 We're very excited today to,

00:03 to welcome you,

00:04 uh,

00:05 here to this May edition of our policy research,

00:08 research talk series.

00:09 Uh,

00:10 this time presented to you jointly by,

00:13 uh,

00:13 the

00:14 Development Economics vice presidency's Research Department,

00:17 as well as DIM,

00:18 the Impact evaluation department.

00:20 As you may know,

00:21 these talks provide us a chance to share the research coming out of uh the World Bank

00:26 with the goal of sharing the findings with

00:27 colleagues inside the institution as well as outside.

00:30 Um,

00:31 I'd like to

00:32 welcome our audience both on Webex and YouTube,

00:35 as well as those of you who made it into the room.

00:38 Thank you for coming.

00:39 Um,

00:40 today I'm pleased to introduce my colleague Dan Roger,

00:42 who is a senior economist in and Program

00:44 manager for DM's governance and Institutions building team,

00:48 as well as the co-founder of

00:51 the World Bank's Bureaucracy lab.

00:53 Uh,

00:54 Dan will be following up on his very entertaining

00:56 and exciting and notorious 2019 policy research talk,

01:00 um,

01:01 which has a similar starting point to today's,

01:03 I think,

01:04 uh,

01:05 you'll see,

01:05 uh,

01:06 where we go,

01:06 I'm going with that.

01:08 Um,

01:09 more data are available on public officials and

01:11 the behavior of public organizations than ever before,

01:13 and Dan has not only been leading the effort to collect these data

01:17 through the bureaucracy lab.

01:19 But also helping to make sense of it through his own research.

01:23 So picking up where he left off his previous talk

01:26 here with us at the policy research talk series,

01:28 Dan will look into new evidence on the design of accountability structures,

01:32 the role of public service culture,

01:34 and methods for better understanding public

01:37 administration.

01:38 And just,

01:39 you know,

01:40 Dan's been with us for a while at the World Bank,

01:43 but just to flag that before he worked at the World Bank,

01:45 he was an economist in the presidency of Nigeria and

01:48 as an associate researcher for UK's Department for International Development.

01:52 So a little bit of broad bureaucratic experience there.

01:56 Uh,

01:56 additionally,

01:56 we're extremely grateful today to have

01:58 Arturo Herrera Gutierrez as our discussant.

02:02 Arturo is the Global Director for the Governance Global Practice

02:05 in the Equitable Growth Finance and Institutions Practice Group.

02:09 A vice presidency,

02:10 uh,

02:10 he put that puts him in a unique position to discuss,

02:13 uh,

02:13 Dan's body of research and what he'll be describing today.

02:16 Arturo's experience,

02:17 uh,

02:18 extensive experience in governance from both

02:20 a practitioner and academic perspective.

02:23 Uh,

02:23 within the bank,

02:23 he served in numerous positions,

02:25 and I don't even know if I have all of them here,

02:27 but I have sector,

02:28 senior public sector

02:29 management specialist and sector manager in the Latin American Caribbean region.

02:34 Uh,

02:34 practice manager in various units include,

02:37 uh,

02:37 for global governance,

02:39 practice in LAC,

02:40 East Asia Pacific,

02:42 as well as the global unit.

02:44 Um,

02:45 between 2018 and 2021,

02:47 he left the bank and held leadership positions in the government of

02:49 Mexico as co-head of the finance team in the presidential transition.

02:54 Uh,

02:54 team,

02:54 uh,

02:55 Deputy Finance Minister,

02:56 and most recently,

02:57 before

02:58 rejoining us,

02:58 uh,

02:59 back at the bank as Minister of Finance and Public Credit.

03:02 So I'll ask Dan to talk for about 45 minutes,

03:05 uh,

03:05 and

03:06 he may need to walk slowly,

03:09 after which we'll hear from Arturo for,

03:10 for 10 to 15 minutes,

03:11 uh,

03:12 and we'll conclude the sessions with Q&A from you in the audience.

03:15 If you have a question,

03:17 please use the raised hand,

03:19 um,

03:20 option in Webex or signal to me in the chat that you do have a question,

03:23 and I'll call on you and hopefully you can ask your question

03:26 yourself.

03:27 Of course,

03:27 if you're in the room,

03:28 just raise your hand.

03:29 Um,

03:30 and if you're following on YouTube,

03:31 please submit your question on the YouTube chat,

03:33 and that'll get relayed to me.

03:35 So just a reminder to everybody,

03:36 we are recording the session,

03:38 and if you're following online,

03:40 please mute when you're not speaking.

03:42 So thank you,

03:43 and with that,

03:43 over to you again.

03:45 Thank you.

03:48 Thank you very much for attending.

03:52 No problem.

03:53 But who's presenting the policy research talk?

03:56 Rogger.

03:58 Daniel Roger.

04:00 I think that's what most of you are coming for,

04:02 so if you want to leave now,

04:03 you can.

04:05 I would like to begin by building on my last policy research talk,

04:09 which was really a,

04:09 a review of the literature on bureaucracy,

04:12 on.

04:14 How you fix bureaucracy through incentives

04:18 and

04:19 and create and I left off on culture.

04:22 But before I do that,

04:23 I wanna take a step back

04:25 and think about the focus of these talks.

04:28 I want to.

04:29 Think about how we influence policy.

04:33 And I want to present to you a thought experiment,

04:35 which I call a kind of model

04:37 of policy influence that summarizes the way that I think about policy influence

04:41 and present what I think recent evidence implies for that model and hopefully

04:46 provide you some motivation for why we should think about this subject today.

04:50 So let's think of policy as frontline decisions,

04:54 decision points

04:56 that

04:56 public officials have to make.

04:58 And these decision points are typically embedded inside a program,

05:02 a program of activity in public service.

05:05 These programs inside sectors like health and education,

05:08 and those inside government administration.

05:12 And so

05:13 for me,

05:14 when I think about public policy,

05:15 I think of the scale of public policy,

05:17 the scale of these decision points as relative to the scale of government.

05:21 So how big

05:22 is public policy?

05:23 How big is government?

05:24 Well,

05:25 there are about 500,000 local governments in the world.

05:28 Something like 183 national,

05:30 various regional,

05:31 like 2000 state and province.

05:34 So now let's think.

05:35 In each of these governments,

05:37 there's something like 10 sectors,

05:38 health,

05:39 education,

05:39 water,

05:40 and so forth.

05:41 And back of the envelope,

05:42 very rough approximation.

05:44 In each of those sectors,

05:45 there's probably 10 programs.

05:46 You can think of them,

05:47 10 budget line items.

05:49 And for each of those programs,

05:51 there's 10 decision points.

05:53 Which village should I operate in?

05:55 What kind of technology should I use to drill a borehole?

05:58 What

05:58 timing should I have?

06:01 Just summing all this up means that every year in the budget,

06:04 there's something like half a billion budget decisions per year.

06:09 That's

06:10 the policy we're trying to influence.

06:13 This is a rather daunting number to me.

06:17 I don't know how many public policy decisions you've influenced in your life,

06:20 but to me this is just an amazing amount of public policy to influence.

06:24 And of course

06:25 some work influences multiple decisions at one time.

06:29 But given the need for validity of any

06:31 research for a particular context or decision point,

06:34 it's still a very substantial number.

06:38 So as an empirically founded institution,

06:40 what we want to know is a little bit about the

06:42 elasticities of the nodes at different points in this model.

06:45 Where can you have impact?

06:49 And so

06:50 We could invest in decision points,

06:52 we could spend our time trying to work out

06:55 which way we should go on a specific decision.

06:57 Or we could say,

06:58 well,

06:59 we want to take a holistic approach to a

07:01 decision point and understand how to operate a program.

07:05 And then of course one could think,

07:06 well,

07:06 let's invest in a sector.

07:09 Work out how to get health going,

07:11 but as soon as you start working with an entire sector,

07:14 you're into issues of administration,

07:15 of how to run the personnel

07:18 and

07:19 the

07:20 organization

07:21 of these programs.

07:22 And so you're into issues of whole of government.

07:27 My big question is,

07:29 how do we affect these different nodes?

07:31 How much does an investment

07:33 actually have influence?

07:35 And so let's start with the classic decision points.

07:38 This is actually quite hard,

07:39 even though we all work on how to improve the decisions of public policymakers.

07:43 What is an average effect that we should expect?

07:47 At DM,

07:48 we've tried to assess the portfolio of IEs that we've done,

07:52 and

07:53 our sense

07:54 is that a good aspirational number is something

07:57 like doubling the effectiveness of a decision point.

08:00 That's the kind of big number.

08:01 Now,

08:02 you know,

08:03 many of the evaluations don't get something this large,

08:06 and there are a few outliers,

08:08 but there's something aspirational about this,

08:10 and we have been able to do this in quite a number of IEs.

08:14 In preparation for this talk,

08:15 I looked at the documents underlying another big

08:18 assessment of development effectiveness,

08:20 the Copenhagen Consensus,

08:22 and that number,

08:22 the idea of doubling.

08:25 Is

08:25 broadly in line with some of the elements of the Copenhagen Consensus.

08:29 And then if you spend your time looking through the JPARL RCT database,

08:32 this idea that you can double development effectiveness

08:35 is not too far from the truth for a body of evidence.

08:41 So if that's what you can do with a decision point,

08:44 what about the administration above it?

08:46 Because in my model,

08:47 all policy is embedded inside an administration.

08:51 So I can now show you evidence from a range of settings on that,

08:54 but I want to start with the graph I showed

08:56 you at the end of my first policy research talk.

09:00 These are

09:01 slides from that talk,

09:02 and the idea that came across is that bureaucracy is fundamentally diverse.

09:07 And actually,

09:08 one of the stylized facts I'd say of public administration empirical work

09:11 over the last 4 or 5 years is that that is true.

09:15 That bureaucracy is just a very diverse place.

09:18 Some things work very well,

09:20 some things fail.

09:21 And when you start to think about this,

09:23 this is ex post kind of obvious.

09:26 There are few systematic forces driving homogeneity in the public service,

09:30 and so.

09:31 If

09:33 Bureaucracy is fundamentally diverse.

09:34 If the administrative setting is such,

09:36 what does that mean for influencing policy?

09:37 Well,

09:39 Let me take you back to the graph in Ghana that I showed at the end of my last graph.

09:42 What this shows us

09:44 is the 35 core public institutions of Ghanaia's

09:48 civil service.

09:49 These are the ministries that we've worked with,

09:52 and they rank

09:54 our data collection over 3 years of how many tasks

09:57 that they were supposed to complete that they completed.

10:00 So

10:01 the Y axis is the proportion completed,

10:03 and just to be clear,

10:04 those underneath the graph,

10:06 those underneath the dots are completed tasks.

10:09 Those above are uncompleted tasks.

10:12 Ghana is in the middle of government effectiveness on the WGI for

10:15 a lot of the kind of clients that we work with.

10:17 This is not unusual.

10:19 But the first thing you think is,

10:21 wow,

10:21 that is a lot of uncompleted tasks.

10:25 And of course,

10:26 the first thing that I think when I see that is that

10:29 the average investment in a good decision,

10:31 a good design of a point

10:34 is going to be heavily mediated depending on where you are in that distribution,

10:38 which ministry you're working with.

10:41 So

10:42 Even if you're working with any but the very best organization in Ghana,

10:46 you immediately lose 40% of your development impact

10:49 if you haven't worked with the administrative environment.

10:52 Just because

10:53 you weren't thinking about that parallel infrastructure for planning.

10:58 And if you work with an organization in the bottom,

11:01 75% of performers.

11:04 You've lost all of the gains in development,

11:06 uh

11:07 development finance that arises from better design because you're losing 50%

11:12 of the tasks that are supposed to be completed.

11:15 The next graph I showed you.

11:18 Was not

11:19 just the organizations,

11:20 but stacked

11:22 on the solid dots,

11:23 which are the ministries are the units inside those,

11:27 those ministries.

11:29 What you see is incredible variation within the same organization.

11:34 So let's take

11:36 A ministry about in the middle of the distribution.

11:39 And see the 3 units that are on this dotted line.

11:42 What you find is that if you're lucky

11:45 and your program or your

11:47 decision point,

11:48 the thing that you've been working on,

11:49 is on the right side of the corridor,

11:51 that unit that gets things done,

11:53 and that's the

11:54 0.8%. Over three years,

11:57 0.8%,

11:58 0 sorry,

11:59 0.80% of tasks were were completed.

12:02 But what if you're unlucky?

12:04 What if when you mentor that ministry and you turn left,

12:07 you,

12:07 the unit you're working with is on the left-hand side of the corridor?

12:10 Well,

12:10 you're down at that 40

12:12 10% point.

12:13 You've lost 90% of your development effectiveness because you're working

12:17 with the unit on the left side of the corridor.

12:20 And what about the one at the end of the corridor?

12:23 That over 3 years produced nothing.

12:28 To me this motivates immensely the idea.

12:31 That

12:32 thinking about public policy isn't just about good design of decision points.

12:37 It's about trying to work with that unit

12:39 or trying to raise these units

12:42 further up this dotted line.

12:45 And as I said,

12:46 is this a Ghana phenomenon?

12:48 Absolutely not.

12:50 Now,

12:50 this is some data from myself,

12:52 from my co-authors,

12:53 from my friends,

12:54 from people in the community.

12:56 We now know that this 0 to 100% distribution across ministries,

13:02 departments and agencies is everywhere,

13:04 everywhere.

13:07 And so let me take you one more step along this journey and say,

13:11 Let me take you to Slovakia,

13:13 where we've been working on productivity data

13:16 and we even have data at the individual level.

13:18 So let me take two tasks.

13:21 So this task

13:22 is business licensing,

13:23 and what you see here is a graph

13:25 of caseload,

13:26 the amount of cases that an individual officer was able to throughput in a month.

13:32 And this is over a multi-year period,

13:33 so these are pretty good,

13:35 uh,

13:35 sense of how productive these individuals are.

13:38 And once again,

13:38 those orange tri

13:40 triangles show you the organization,

13:42 the district average.

13:45 Huge amounts of variation across districts,

13:47 but even more amongst individuals.

13:49 And this is true across all the different types of tasks that we look at.

13:53 On the right is environmental regulation.

13:56 People,

13:57 officers looking at cases of environmental regulation.

14:01 So in Slovakia,

14:02 now it's not just turning

14:04 to the right unit

14:06 in the right

14:07 organization,

14:07 it's turning to the right person.

14:10 The public administrator is going to hugely

14:13 mediate your capacity to convert better decision points

14:17 into development impact.

14:21 And so when I think about this model,

14:24 I think to myself,

14:26 Well,

14:26 then surely we should be working on how public administration works,

14:30 as well as working on the good design of individual projects,

14:34 individual decision points within those projects.

14:37 But I also think about the scale.

14:40 Half a billion decision points a year,

14:43 how are we going to affect that?

14:44 Working on individual decision points is going to be tough

14:48 to influence policy writ large.

14:50 But if we

14:51 influence the machine

14:53 that defines those decision points,

14:55 maybe we have more of a chance.

15:00 It was interesting to hear Esther Duflo's reflections

15:04 two weeks ago at the Dime Impact Week,

15:07 when she

15:08 said summarizing her entire body

15:11 of experience and work.

15:13 But what matters enormously in generating

15:15 successful development impacts and knowledge are

15:17 the details of implementation that no one can think of in advance.

15:21 And they are solved in partnership with our

15:25 often government partners.

15:29 And we also all knew this.

15:31 Anyone who's ever tried to implement a project or program in

15:34 the field knows if you get the right field coordinator,

15:38 manager,

15:39 individual public official that you're working with,

15:42 everything goes right.

15:44 But if you get the wrong one.

15:46 You never do the evaluation.

15:48 You never roll out that project.

15:50 The operation doesn't work.

15:52 But we know this,

15:54 and yet it is not at the center of how the bank works,

15:57 nor at the research that we do.

15:59 If public administration is so mediating and here's all the data to show it,

16:04 then surely it should be something we take very seriously.

16:08 So that's what I hope to do in the slides next.

16:10 Now,

16:10 before I start,

16:12 I have heard many times,

16:13 Mr.

16:13 Bond,

16:14 you are forgetting about politics.

16:16 All bureaucratic outcomes are surely just defined by the political settlement.

16:22 So

16:23 I'm gonna push back on this idea

16:26 partly

16:27 reflecting on the model I've just showed.

16:30 I do believe,

16:31 as I said in my first

16:33 policy research talk,

16:34 that political settlements clearly matter for aspects of

16:36 bureaucracy and they're part of the development process,

16:39 but

16:40 At the scale that I've just shown you,

16:42 half a billion decision points a year,

16:44 it feels hugely,

16:46 implausibly demanding that politicians are influencing

16:49 every single one of those settlements,

16:51 every single one of those decision points.

16:53 Personnel,

16:54 informational,

16:54 cognitive,

16:55 and other resource constraints likely apply to politicians everywhere,

16:58 and much

16:59 and make it much more likely that there are

17:01 areas of passive inefficiency that have real welfare consequences.

17:05 And this will be true of individual decision points,

17:07 programs,

17:08 and whole sectors.

17:09 And the paper that I think articulates this best is Bandiera Etal looking Italian

17:13 procurement.

17:14 They even split

17:15 the waste they see into passive inefficiency and active inefficiency.

17:20 And what they find is that 85% of inefficiencies are passive.

17:27 That is an exciting number.

17:30 Because it,

17:30 it means we can go and search

17:32 for the 85% of inefficiencies in administration,

17:36 or 60% or 30%.

17:38 That's a lot of work for us to do.

17:40 It's still indicative of lots of room for a

17:43 better administration to make a difference to people's lives.

17:47 But let me discuss how I think.

17:50 We should go about doing this,

17:52 and it takes off from the last words I said in my policy research talk,

17:56 is that bureaucratic culture fundamentally mediates effectiveness.

18:00 Now there's fantastic qualitative work coming out on exactly this,

18:04 and it points to that functional bureaucracies

18:06 are characterized by strong interpersonal bonds,

18:09 joint accountability,

18:10 not to us but to each other.

18:13 And so the question those of us

18:15 who work in more quantitative and intervention-based work

18:18 want to understand is,

18:19 well,

18:20 What are the interventions we might

18:23 actually

18:25 intervene in service

18:27 to get government functioning with these time team dynamics?

18:30 So I'd like to present to you 3 impact evaluations to start that

18:35 indicate where I think we should go.

18:38 I'm going to begin by assessing perhaps the

18:40 most common notion of an effective bureaucratic culture,

18:43 that of centralized accountability.

18:46 The idea that a senior member of government,

18:48 a strong man,

18:49 and it is almost always a man,

18:51 can drive public sector performance from the center.

18:55 This is

18:56 an often.

18:58 Talked about idea that this is how you change the culture of the public sector.

19:04 So my co-authors and I assess the impact of a

19:07 showcase centralized accountability system implemented

19:10 in Punjab province in Pakistan.

19:12 And so the scheme allows us to say something about what happens when you try to drive

19:18 performance culture from the center using fantastic data.

19:21 An amazing data system was put into place.

19:24 And our core interest

19:25 is whether senior management can sort of

19:27 force an improved public sector performance,

19:30 having been empowered by this management information structure.

19:34 So what we do is we take a 6 year eva we evaluate a 6 year period

19:38 in which the Chief Minister of Punjab was given

19:41 detailed data on how each educational district was performing

19:44 and alerted to the failure of an individual bureaucrat.

19:48 And then that failure would cause great de facto punishment.

19:53 We assess

19:55 the impact

19:56 at initiation

19:57 and then

19:58 on bureaucrats

19:59 over these six years who are flagged red and those who are not flagged.

20:04 And we capitalize on the fantastic administrative data

20:06 on a whole range of great things,

20:08 student-teacher presence,

20:09 learning facilities,

20:11 bureaucrat careers,

20:12 staffing,

20:12 finance.

20:12 The data was fantastic.

20:15 Was using it for accountability like this the right way to go?

20:19 This is what the Chief Minister saw.

20:21 They saw a dashboard

20:23 of colors of green and amber,

20:26 and then those dreaded reds.

20:28 And the bureaucrats did not want to be classified as

20:32 the cause of a red.

20:34 Now,

20:35 what we can do because the data is so rich,

20:36 is we can go down to the school level,

20:38 and we can look at schools that were in those greens,

20:41 but just in those greens.

20:44 And those

20:45 reds,

20:46 but a school that was just in the red

20:48 and caused by other schools in the district.

20:51 These two schools that we're going to assess are

20:53 exactly the same in terms of their history,

20:54 in terms of the shock that they received.

20:58 And so we can do a relatively clean comparison

21:00 of what happens to the school outcomes when the bureaucrat

21:03 is shouted at,

21:05 is has reputational damage,

21:07 has all that de facto punishment.

21:11 And I'm going to do so by telling,

21:12 walking you through what happens to the average school.

21:16 So what you see here

21:17 in the,

21:18 let's firstly take the two solid lines.

21:22 Is

21:22 schools either side of that threshold,

21:25 and the red ones are those that

21:27 just got into red and the blue that stayed on the side of green.

21:31 What you see is both of them have a shock.

21:34 This is teacher presence.

21:35 The teacher got sick,

21:36 the teacher couldn't come to school in that day.

21:39 And so

21:40 teacher presence falls.

21:43 And so now we want to know in the next period where

21:46 the red is punished.

21:48 Do we see a different trajectory?

21:50 Do we see the more accounted school,

21:52 do we see it change the way it operates?

21:56 As you can see,

21:57 absolutely not.

21:59 Schools that are punished revert

22:01 to the original pathway in exactly the same way.

22:05 This

22:06 driving through performance management at the center does nothing,

22:09 it achieves nothing.

22:11 And then I can also show you the same thing for all schools,

22:13 which are the dotted lines.

22:15 The school reverts to where it would have been otherwise.

22:20 We don't find impacts on any margin of centralized accountability,

22:24 teacher presence,

22:24 student attendance,

22:25 physical facilities,

22:27 any of the scores,

22:28 nor do we find on the activities of bureaucrats.

22:34 So

22:36 Is this a rogue finding from Pakistan?

22:39 No.

22:40 The

22:41 work that I just showed you was embedded in a six-country case study

22:45 where we looked at the same things,

22:47 researchers looked at the same things across 6 countries.

22:50 And

22:51 Martin Williams and Claire Lever were here a couple

22:53 of weeks ago to showcase the endline results.

22:57 And what they argue is across all countries,

23:00 data is not so useful as an

23:02 indicator for allocating top-down rewards and sanctions.

23:05 Rather,

23:05 there are huge potentials for organizational learning.

23:09 But this is not always prioritized.

23:11 In fact,

23:11 very rarely prioritized.

23:13 It's very,

23:14 very rarely integrated with mainstream civil service routines.

23:19 So what my co-authors and I did to show what

23:22 could have been done with that data.

23:24 Is we

23:26 follow Alessandro Fennizia's work identifying the

23:28 talents of individual public officials,

23:31 and

23:32 we take head teachers

23:33 and we give each of them a talent score in

23:36 their ability to bring teachers to school,

23:39 in their ability to keep student attendance up.

23:42 And what we find is that can move the needle.

23:44 If you get a talented head teacher to your school,

23:48 that's what changes teacher presence.

23:50 Let me focus in on the graph below for a moment.

23:53 What this is is a scatter plot

23:55 for the talent indices that we create for every head teacher.

23:59 So

24:00 what you see on the

24:02 teacher presence column.

24:04 Is

24:05 all those teachers' scores,

24:07 but then cross-correlated with how they did in other one.

24:10 So the purple shows us

24:13 how well does a teacher able to bring teachers to school,

24:16 but also to raise functional facilities of a school.

24:19 And what's fascinating is that

24:21 that is a basically random scatter graph.

24:24 Whether a teacher is good at

24:26 getting teachers to school

24:28 or good at functional facility maintenance.

24:32 It's not that related.

24:33 Now,

24:34 it's quite related,

24:34 you can see with scores,

24:36 maybe as we'd imagine,

24:37 you

24:38 are good at getting math scores,

24:39 maybe you can get English scores up.

24:41 But what's fascinating here is that we can use data

24:45 to identify the individual differential talents of public sector workers.

24:51 The comparative advantage of that coarse but large scale data in the public sector

24:56 is to identify underlying parameters that no

24:59 individual bureaucrat could do on their own.

25:03 No public official knows where they are in the large end distribution.

25:08 Most of those public officials in the top

25:11 tiers of

25:12 being able to bring kids to school,

25:14 being able to get teachers to school,

25:16 didn't know

25:17 that they were one of the best in Punjab.

25:19 So if we give them that data back,

25:21 well then

25:22 we can complement management,

25:24 their management,

25:25 and the management of people higher up the chain

25:28 with very targeted data.

25:33 So a call will come,

25:35 surely you're not saying we should have no accountability in the public service.

25:38 That's right.

25:39 What I just talked about was centralized strongman accountability.

25:44 But how about being accountable to each other in our departments,

25:47 in our organizations?

25:49 The classical way that the bank has done

25:52 that is through implementing appraisals like the OPE.

25:56 It's where you set targets.

25:57 This is what I'm going to achieve this year.

25:59 And so with some co-authors,

26:01 I spent the last 6 or 7 years

26:04 studying what happens when you implement an appraisal system in Liberia.

26:08 Liberia

26:10 is a country that had

26:12 a ravaged

26:14 public service after the civil war.

26:16 It was a place where people did not come to work,

26:19 where people were unhappy,

26:20 where people turned over.

26:22 It was

26:23 very difficult to manage.

26:26 And so

26:27 our interest here is whether we can encourage the uptake of the appraisal

26:31 system as the foundation of public service structure and see what happens.

26:36 Well,

26:36 we implement two treatment arms in

26:38 collaboration with Liberia's civil service agency.

26:41 One where we try to roll out communications,

26:43 training and support to get the appraisal up and running,

26:47 focusing on the role of senior managers and peers.

26:49 But today I want to put them together and just say what happens

26:52 when you encourage an appraisal system you put in in this fragile environment.

26:58 We collected data on the impact of rollout over the next 2 years on attendance,

27:02 management,

27:02 quality,

27:03 experience of the state.

27:05 And what we find,

27:06 and here's a regression table,

27:08 and I want to show you that because

27:11 People start turning up more.

27:14 These numbers are attendance figures.

27:17 Because we gave some accountability,

27:19 because there is some structure,

27:20 people start arriving more

27:22 and

27:23 voluntary turnover goes down.

27:25 People stop

27:26 leaving their units as much.

27:29 They're also more satisfied.

27:30 Here's some satisfaction measures.

27:33 Satisfaction goes up 8% points.

27:36 And when we ask,

27:38 you know,

27:38 why is it you don't want to leave the unit,

27:40 it's to do with their intention of moving up through the service.

27:46 And so in Liberia,

27:47 an environment in which one would imagine it's very difficult to get control,

27:51 build structure,

27:52 we find detectable effects,

27:55 big detectable effects.

27:59 When we did qualitative interviews.

28:02 The individual said the system is being introduced

28:05 in our scope of work is actually good.

28:06 It helps motivate us.

28:08 It encourages us a lot.

28:10 You know,

28:10 there are a lot of things behind it.

28:15 But we don't want to just create structure so that people turn up to work.

28:19 We want people to innovate.

28:21 We want people to strengthen public officialdom.

28:25 And so in Ghana,

28:27 with co-authors,

28:29 we study exactly this idea,

28:31 a culture of innovation.

28:32 How do you imbibe a culture of innovation so

28:34 that public servants can build the service themselves.

28:39 This is based off a bureaucracy lab that I told you

28:43 was starting in my last public service policy research talk,

28:46 and

28:48 What's wonderful is it's still going and they're doing stuff.

28:50 In fact,

28:50 we have limited to do with them now,

28:51 but they're still working.

28:53 And this is the initiation of that,

28:55 that,

28:56 uh,

28:56 bureaucracy lab.

28:58 But specifically what we wanted to do with them is say,

29:02 You know,

29:03 can we change the way that you train

29:05 public officials to make them more public service

29:09 interested,

29:09 excited?

29:10 Can you make them better

29:12 initiators of new ideas and generate a better public service?

29:18 So we collaborated with the Office of the Head of Civil Service,

29:22 and all we did

29:23 was we took this 2 week standard training.

29:25 Now,

29:25 I'm assuming that most people in this room

29:27 have done some World Bank public service training.

29:29 It is a core way that we build capacity in the public service

29:33 across the world,

29:34 but more importantly,

29:34 it's a

29:35 very large part of the bank's capacity building efforts.

29:39 So we just added a module,

29:41 a single module,

29:42 where we worked with people to diagnose and

29:45 solve problems based on what they'd learned.

29:48 To innovate from the bottom up,

29:49 to conceptualize change,

29:51 we role played,

29:51 we gave them a platform to take what they'd learnt back into the service.

29:56 And here we again implement two treatment arms,

29:59 one where we take individuals into a classroom and make

30:01 them into individual innovators in their unit or department.

30:05 And another where we go to the unit or

30:07 department and we try to get everyone to innovate.

30:10 And I want to split the results I show you

30:13 because the results are so interesting along this line.

30:17 So we collect data on the impacts of these trainings over the next 2 years on culture.

30:21 Can we change culture,

30:23 administrative process,

30:24 task completion rates,

30:25 and

30:26 thank you KCP for making that last two things possible.

30:31 So what do we find?

30:32 Again,

30:32 I'm showing you regression tables.

30:35 Because I want to show you how strong the effects are here.

30:40 So if you have this innovator training,

30:42 what's fascinating is

30:45 that

30:45 having a single innovator.

30:48 Improves overall division culture.

30:52 What you have here is claims about division culture from a civil servant survey,

30:56 and individuals say that the teamwork climate goes up,

31:00 performance climate,

31:02 fostering new ideas.

31:04 The

31:05 the entire unit's quality of culture is improved.

31:09 And when we do it with

31:11 trying to train everyone together,

31:13 when we go out

31:14 to the entirety of the department,

31:17 we find no effects.

31:21 Now,

31:22 what about idea sharing and generating,

31:24 generating improved processes?

31:27 Well,

31:27 here we,

31:27 we see is that those who are in the classroom training where we created innovators,

31:32 well,

31:32 they feel that there's now more freedom to express those

31:35 ideas and the frequency of meetings on process increases.

31:39 And yet,

31:40 the best evidence we have from

31:42 the classroom training where we train everyone together.

31:46 Is null to maybe even negative.

31:51 And finally,

31:52 does this have an effect on process itself?

31:56 Yes.

31:57 And not just that,

31:58 if you look at the column at the end,

31:59 do you remember those task completion rates I showed you at the very beginning?

32:04 They increase by 11% points.

32:05 It's noisy,

32:06 but something's there.

32:09 We don't find any similar effects when you try and train everyone together.

32:14 Why?

32:16 You know,

32:16 we have some quality evidence.

32:17 With your superior in the training,

32:19 it was not beneficial,

32:20 said one for feedback of individual from from team training.

32:23 You have to be cautious about the superior subordinate relationships.

32:27 I did not see the benefit and I could not voice my feelings.

32:32 You know,

32:32 each of these impact evaluations surprised me.

32:36 And

32:37 they underline it.

32:39 How much

32:40 and how little we know about how the public service functions.

32:44 We've probably been training entire units.

32:48 For 40 years.

32:50 Have we ever created innovators?

32:52 Probably sometimes.

32:54 Did we do it in the right way?

32:57 You know,

32:59 administration can be changed,

32:59 it seems.

33:01 And for the better.

33:02 And even small but well-designed interventions can have impact.

33:05 And again,

33:06 that's expost obvious because we're probably so far from the production frontier

33:10 that small tweaks can push us quite a long way towards it.

33:14 Yet what works and doesn't is so poorly understood.

33:17 And for an organization

33:19 whose

33:20 primary intention is to influence policy,

33:24 to me,

33:26 This is an urgent need for better information.

33:30 You know,

33:31 I'm not asking us to respond to any of the IEs that I've done.

33:34 I would not want us to respond too intensely to any single impact evaluation

33:38 or set of findings,

33:40 but surely we'd want to update our priors and invest

33:43 in this kind of knowledge.

33:46 Bureaucracy does seem responsive to interventions that build up

33:49 professional features of the service like clarity and duties,

33:52 identification of talents,

33:53 responsibilities to improve procedure,

33:55 and yet in the economics profession,

33:57 there is almost no impact evaluation on these things.

34:02 So is another world possible?

34:04 I think it's vital

34:06 because we only have 6 years to achieve the SDGs.

34:08 For those of us who think about this sort of thing,

34:11 we've basically run out of time

34:13 to introduce new programs and policies with uncertain impacts.

34:16 What we do have time is to go and find the talented managers

34:19 and the talented public servants and put

34:21 them where they're needed most strategically.

34:25 We need a technology that has immediate and sustainable effects.

34:28 You can move a manager in a month.

34:31 Something that can feasibly impact those half a billion

34:33 policy decisions where a dollar has catalytic impacts.

34:36 And of course,

34:37 beyond the SDGs,

34:38 there are going to be global challenges that require

34:40 an interconnected and long-term approach to policy influence.

34:43 The battle will not stop here.

34:45 And what program can achieve that?

34:47 My opinion is that it is one focused on analyzing and strengthening government.

34:53 My first policy research talk

34:55 sketched out

34:57 how

34:58 the

34:59 my public service is a unique and complex place.

35:02 You should go and watch it,

35:03 it's good.

35:06 It showed us how microdata is so important because in such a diverse place,

35:10 making a single statistic for the entire of a government.

35:15 Doesn't tell us too much.

35:18 But what I'm saying now is we have to act on that information.

35:21 We need to use that information

35:22 to improve government.

35:25 And we're not going to influence public policy unless we do so.

35:29 You know,

35:30 there are likely to be substantial passive inefficiencies,

35:33 the resolution of which have substantial welfare gains.

35:36 So why are we not changing the narrative of

35:38 the World Bank towards supporting government and its officials

35:42 to allow them to work more effectively on their decision points?

35:48 What do we do now,

35:49 James?

35:51 Well,

35:53 Michael gets a,

35:55 a starring role.

35:56 He has told us

35:58 for years.

36:00 What are you doing for the median official?

36:02 His pioneering work trying to push us to

36:04 think about the implementation side of our work

36:07 is exactly where we need to go,

36:09 and we can have different flavors.

36:11 The work that I do is different to the work that Michael does,

36:14 but I think we both agree that the question we should be answering is,

36:17 are you supporting a public official to be their best professional selves?

36:21 That's a question we should all ask ourselves every day.

36:25 My answer is government analytics.

36:28 It's to repurpose administrative and survey data from

36:31 within government to improve public sector functioning.

36:33 Release the service.

36:36 To improve itself.

36:38 And this offers us.

36:40 A

36:41 distinct microdata-based way to improve state capacity.

36:46 It enables governments and development partners to detect areas of improvement.

36:49 I've shown you how just descriptives show you where all of the least

36:54 effective agencies are.

36:56 I've also shown you that tweaks to that administrative data

36:59 allow us to target where talent needs to be supported.

37:03 We can work government-wide,

37:05 but more likely we can work agency by agency.

37:08 We can draw on a range of sources,

37:10 so we're not just taking one statistic

37:13 and using that to drive our effects.

37:16 But.

37:19 It means I have to face a very hard reality.

37:24 It turns out I am not James Bond,

37:28 and not just because of the good looks.

37:31 It's because of the immensity of public policy.

37:34 There are just

37:37 Too many decision points,

37:39 too much action for me to be able to save the world.

37:45 I'm afraid that's true more generally,

37:46 we are not James Bond.

37:49 We're not going to save the world by

37:50 shooting down decision point after decision point.

37:52 There are just too many of them out there.

37:54 What we need is an army of capable.

37:58 Committed

37:59 and not totally constrained.

38:02 Public servants,

38:03 our James Bonds

38:05 all over the world.

38:08 So where does that leave us?

38:12 Well,

38:13 we're in this building or in a country office.

38:16 So we're back at headquarters.

38:20 Once we're humble enough to realize who the real heroes are,

38:22 we can at least put our lab coat on.

38:25 And be some sort of cue.

38:28 One slightly nutty analyst whose intention is to build

38:32 tools to support field operatives and in our case,

38:34 public servants.

38:36 That's what we can do.

38:38 And if that is our intention,

38:39 then I think it changes what the World Bank should be doing in policy and research.

38:44 Let me tell you a little bit about how I think we should play the role of Q.

38:51 We should ask public officials more,

38:52 the median public official.

38:55 What it is

38:56 that they're facing today and how can we help

38:58 with the vast amounts of data that we have.

39:02 We can be a conduit to the information that they need,

39:04 not that we think they need.

39:05 We can use bank projects and financing as

39:08 conduits for building the capabilities of public administration,

39:11 but based on rigorous

39:12 evidence of how to do it.

39:14 We can provide officials with inspiration.

39:18 Tools and comparative data to help them make better choices,

39:21 and we can work hand in hand to make them better analysts of public policy.

39:25 I think that's what DM is trying to do.

39:27 We are trying to do that in our

39:29 iterative way.

39:31 You know,

39:32 we have this thing called trial and adopt.

39:33 It works directly with public officials on specific programs,

39:37 specific problems and evaluations,

39:38 and therefore we have an incentive to listen,

39:41 to support capacity building,

39:42 to help

39:43 build data systems inside government.

39:46 Otherwise,

39:47 nothing will be adopted

39:48 when we suggest our mid-course corrections.

39:52 Ariana says it,

39:53 that we focus on skills development and behavioral

39:56 change to support the deployment of development ideas

39:59 as an opportunity to build local capacities for linking data and analysis.

40:04 My flavor of DM has been the bureaucracy lab.

40:08 Zahid Hassanein and I have been building it for years and

40:11 we're still going.

40:13 We're building a collaborative space for deck,

40:16 research

40:16 and operations,

40:17 the governance practice.

40:20 But I think we can do more,

40:21 we can

40:22 present

40:23 practical best practice.

40:25 Myself and Christian Schuster have.

40:28 Been working on this with

40:30 Galli and others to create a government analytics handbook.

40:34 We can share comparative data.

40:36 We're going to launch,

40:37 thank you Aisha for all the hard work,

40:39 the Global survey of public servants,

40:40 the first comparative data on what the public service looks like.

40:45 We can place data in the hands of public managers about their units.

40:49 We can benchmark institutions inside government and show

40:53 how do you do compare with your peers,

40:55 and sometimes because this is demanded,

40:57 we can even show them national benchmarks for governments all around the world.

41:02 And we should make everything we do.

41:04 Completely free

41:06 and completely reproducible,

41:07 which is what I intend to do with every

41:10 measurement piece that I ever do.

41:12 And people like it.

41:13 My favorite,

41:14 of course,

41:15 is back with Nana in Ghana,

41:17 who said that the survey we did with him

41:19 was the first time he'd truly seen his service.

41:23 And why are we not running more experiments inside government?

41:26 Experiments are good ways to learn.

41:27 Let's use them

41:29 to understand how government works.

41:33 Does any of this work?

41:35 It sounds good.

41:38 So what Ravi Samani and I did in Ethiopia was to randomize

41:43 information briefings like I'm suggesting.

41:45 We gave it to some public servants and not others.

41:48 Ethiopia's a a a very important

41:51 counterpart to the bank.

41:54 But when you actually ask public officials

41:56 about the people that they're working on,

41:59 they get it very badly wrong.

42:02 50% of officials that we ask make errors about

42:04 their underlying objective measures like the population they're serving,

42:08 the education levels,

42:09 the number of education.

42:11 They get

42:12 errors 50%

42:14 or more.

42:16 And so what we did was we

42:18 provided a random subset of government officials with

42:22 An information briefing

42:25 And we then collect

42:26 data on the accuracy of their beliefs afterwards.

42:30 Now we do this across the whole of government.

42:32 Now what would be the classic response?

42:34 Well,

42:34 of course,

42:35 it's local governments.

42:36 They're the people who know their environment.

42:39 Actually,

42:40 the evidence briefings improve the accuracy of their beliefs

42:44 by an amount similar to decentralization.

42:46 Decentralization by 1/3 of a standard deviation,

42:50 the information briefs by 25%.

42:53 My colleague Aidan Coville and his co-authors have started to work on this,

42:57 but

42:58 if we're trying.

43:00 To update public officials' beliefs,

43:02 why is there so little evidence in the bank on how to do this?

43:10 I want to shout out

43:11 Juan Santini.

43:15 Who with his co-authors

43:16 did a great piece of work where he informed mayors about research

43:20 and

43:21 is able to increase

43:23 the adoption of those policies by 10% points.

43:27 What I'm suggesting,

43:29 what we've been doing,

43:30 it does work.

43:33 But only when it's fully embedded in an

43:35 understanding of the administration it's trying to affect.

43:39 If we're trying to influence policy,

43:41 those half a billion decision points.

43:44 It's going to be hugely mediated by exactly these things.

43:50 I'm quoting myself

43:53 Our job is to

43:54 not to constantly seek out ways to punish public servants when they get it wrong,

43:58 and to be honest,

43:59 we have limited ways of doing so.

44:01 Rather,

44:02 our best chance is to invest in helping them get their administration right

44:06 to give them the freedom to succeed.

44:09 To ladies and gentlemen,

44:11 You

44:12 have a license to respond.

44:14 If anything of what I've said is

44:17 resonating,

44:17 is interesting.

44:21 Well then,

44:22 firstly,

44:22 think of public servants that you work with as James Bond.

44:26 Give them a sense of autonomy,

44:27 a high mission orientation,

44:29 help them search out information on a case by case basis

44:32 and support that culture of strong professional relationships.

44:37 And I'm now going to try and give each of you some thoughts on what we might do.

44:42 And I'm going to separate them into the 3 audiences I think these talks try to talk to.

44:47 Researchers.

44:48 Those of us who try to be on the

44:51 fence or the

44:53 bridge between

44:54 policy and research,

44:56 and then operations.

44:59 Hopefully,

44:59 members of DEC can remember DEC.

45:02 Describe,

45:03 engage,

45:03 and cross-randomize.

45:05 When you're doing work

45:07 in any field,

45:08 try to measure the administration that it involves,

45:11 that it engages with,

45:12 and add that to global knowledge,

45:14 don't let it just

45:15 fall away in your appendices.

45:19 Engage with your counterparts,

45:22 not

45:22 only on the project that you're doing,

45:24 but ask them how can my data help you more generally.

45:27 And cross randomize.

45:28 If you're doing work,

45:30 then try to bring in an element of

45:33 the impacts of bureaucracy,

45:34 because given the data I've shown you,

45:36 it's likely to mediate it when it gets into public policy anyway.

45:41 To my lab scientists,

45:43 LAB

45:45 link,

45:46 anchor and best practice.

45:48 You know,

45:49 public service is not a place where information flows freely.

45:53 It's not a market,

45:54 an atomistic market,

45:56 and therefore improve information flows between

45:58 public sector actors who should meet

46:00 in the Ethiopia,

46:01 Estonia,

46:02 and other case studies that Aidan and I have done,

46:05 tacit knowledge,

46:06 who you talk to matters hugely

46:09 and therefore,

46:10 who are you going to link up today?

46:12 Anchor,

46:13 generate or support a community of practice that

46:16 shapes public service norms and best practice.

46:19 Support public officials' ability to access that best practice in

46:22 a way that easily translates to their service environment.

46:27 Operations,

46:29 although.

46:31 I feel shy

46:32 talking about something where there's so much expertise in the room.

46:36 But OPS

46:37 observe,

46:38 practice and strengthen.

46:41 Make measures of the administration part of your project

46:44 as they might mediate your success.

46:47 Put

46:47 one performance indicator based on public service.

46:52 Practice,

46:53 trial different approaches.

46:55 Trial and adopt.

46:56 And share your results with us

46:58 and strengthen.

47:00 Always think about how your operation is

47:02 strengthening the administration you're working with,

47:04 so that the next operation might not be as mediated as yours will be.

47:11 People must be very upset that I've said they might not be able to be James Bond.

47:15 But you can be actually.

47:21 There is of course one place that World Bank staff can be James Bond.

47:25 And that's at the World Bank.

47:29 You know,

47:30 there's no administration actually that has been more resistant to my attempts.

47:35 To undertake analytics on it than that of the World Bank.

47:38 I've been rejected at every turn.

47:41 But this is in fact the one place where we could be James Bond.

47:44 This is the administration we're most equipped to truly change.

47:48 But we have to focus,

47:49 focus on changing the administration of the bank.

47:52 Working on individual decision points will not be enough.

47:57 What little data I have

47:58 on.

48:00 This place

48:02 that I've been able to gather

48:03 shows that there's a set of unique challenges here.

48:07 The bank's employee survey has a few questions

48:09 that overlap with the questions the rest of the

48:10 world asks and that my collaborators and I have

48:13 harmonized in the global survey of public servants.

48:15 Here's a few of the overlapping questions and indicators of where we sit

48:19 in the global distribution.

48:20 Although because so few countries ask questions like ours,

48:23 there's only a few comparators.

48:26 We are,

48:27 as many of you would agree,

48:28 a motivated bunch.

48:30 We sit at the top of the distribution for a number

48:32 of questions related to how motivated we are by our work,

48:35 both deck

48:37 and the group as a whole.

48:39 However,

48:41 When you look at how we organize ourselves,

48:43 things look worse.

48:45 We are at the bottom of the distribution in terms of how we collaborate.

48:49 And how effectively we build pathways to capitalize on the bank's talent.

48:56 We should change that.

49:00 Sometimes we hold at arm's length the idea

49:02 that the World Bank is a public administration.

49:05 It is.

49:06 And thus everything I've argued here today

49:08 is applicable to the bank's own administration.

49:13 Well,

49:14 it's time to wrap up.

49:16 I hope something that I've said here today resonates with you.

49:19 If it does,

49:20 please think about it.

49:22 I believe that

49:24 fixing bureaucracy is something.

49:26 We should all

49:27 be part of.

49:28 James Bond will return,

49:30 he always returns.

49:32 How are you going to help him save the world?

49:35 Thank you.

49:43 Super.

49:43 Thanks,

49:43 Dan.

49:46 Yeah,

49:46 we,

49:46 we didn't,

49:48 there are no martinis outside,

49:50 just

49:50 set your expectations.

49:52 Thanks,

49:52 Dan.

49:53 Uh,

49:53 over to you,

49:53 Arturo.

49:58 Um,

49:59 so I'm not so sure where to start.

50:02 Uh,

50:03 uh,

50:03 really interesting.

50:04 Let me actually start on the,

50:05 on,

50:06 on the last part because I,

50:07 I think we are.

50:09 We are looking to some of the issues from,

50:11 from a similar perspective.

50:13 Uh,

50:14 we,

50:15 uh,

50:15 in governance,

50:16 we,

50:16 we work with institutions across the world,

50:19 the world,

50:19 and we try to fix things,

50:20 uh,

50:21 uh,

50:23 how to set up their systems,

50:24 how to make,

50:25 uh,

50:25 they,

50:26 their human,

50:27 manage their human resource,

50:28 how to manage their budget,

50:29 etc.

50:30 And,

50:30 and as such,

50:31 we,

50:31 we have experts who,

50:33 who work

50:34 on the,

50:34 on the civil service side like your

50:37 colleague Saheed.

50:38 Uh,

50:39 and also people who work on,

50:40 on,

50:41 on gov tech issues in how to manage the information produced by the government

50:46 and uh I,

50:47 I would say 3 or 4 months ago I had the same concern that like that you were having,

50:52 and I,

50:52 and,

50:52 and,

50:53 and actually it was not an original idea of myself.

50:56 Let me tell you how I come to.

50:59 To a to a similar point of should we be looking at how to

51:02 help this institution,

51:04 uh,

51:05 to work better,

51:06 right?

51:06 And,

51:07 and,

51:07 and I remember

51:08 I,

51:09 I was remember a failed

51:11 attempt that was intended 14 years ago,

51:14 um,

51:15 you know,

51:15 during,

51:16 during the,

51:17 uh,

51:18 the financial crisis in 2008,

51:20 there were several banks who were the first who were hit.

51:23 The first one was Bear Stearns.

51:25 And uh,

51:26 like many,

51:28 like many banks they have

51:29 people

51:30 who are experts on

51:32 two different kinds of financing.

51:34 The ones who,

51:35 who deal with the clients were helping clients to raise money

51:39 into a different

51:40 spectrum of,

51:42 of,

51:42 of,

51:42 of challenges,

51:43 and there were people who were raising money for barristers,

51:46 their own treasury.

51:48 So when,

51:48 when the crisis hit

51:50 and it was incredibly difficult for,

51:52 for,

51:53 for investors to raise money,

51:54 somebody say

51:55 why don't we ask

51:57 our client face team,

51:59 our investment bankers

52:00 who usually go and raise money for a company which is in uh

52:05 passing troubles

52:06 to raise money for us.

52:08 So they asked their,

52:09 their,

52:10 their in-house experts

52:12 to,

52:12 to look for a,

52:13 for an internal pro problem,

52:15 and that's what we're proposing here so.

52:17 Interestingly enough,

52:18 I asked my team 3 or 4 months ago to do the same,

52:21 so they work,

52:22 and I asked them to prepare a 2-pager

52:25 about how to fix

52:26 the batch rotation and the bank

52:28 and how to get the information that was there.

52:31 So,

52:31 so were you,

52:32 I invite,

52:33 I invite a Sahid which uh was part of them,

52:36 and we prepared the two pager.

52:38 I think,

52:38 I think we can,

52:39 we can help to solve.

52:42 We came from a similar angle.

52:43 We probably,

52:44 we,

52:44 we're probably young because I sent that the two pager to somebody and

52:48 I'm not sure if they have an open it,

52:50 uh,

52:51 but,

52:51 but,

52:51 but,

52:52 but that's that,

52:53 but

52:54 I mean,

52:55 as I said,

52:56 we are,

52:56 we are addressing some of the issues from a similar concern.

53:00 Having said so,

53:01 I have to confess that when I said,

53:03 uh,

53:03 I saw that the cut theory of development,

53:06 I,

53:06 I thought it was about

53:07 Tobin's queue.

53:09 And,

53:10 but this,

53:10 this is,

53:10 uh,

53:11 so I,

53:12 I wasn't sure how to,

53:14 how to,

53:14 to grasp it.

53:16 Let,

53:16 let me,

53:16 let me now,

53:17 uh,

53:18 move to some aspects which I think um.

53:21 Uh

53:24 Required to probably

53:25 to to to probably uh contextualize it.

53:28 uh I,

53:28 I want to start with one of the

53:30 of your statements through through the presentation you said

53:34 uh we urgent need,

53:36 we have an urgent need for better information.

53:39 And that I couldn't

53:41 I I couldn't agree more.

53:43 Now the question is what kind of information?

53:46 And um

53:48 it could be very complex information if the problem is really difficult to crack,

53:52 but it could also be very basic information.

53:56 And in,

53:56 in some aspects,

53:57 what we are missing

53:59 is very basic information.

54:00 Just,

54:01 just trying to do

54:02 this descriptive statistics it's gonna be,

54:05 is,

54:06 is gonna be,

54:06 is,

54:07 is gonna be

54:08 uh uh really important.

54:10 And then.

54:13 A different question is what we need that information for.

54:17 And what I found very commonly both.

54:21 In the ply work

54:22 and here.

54:24 Is that

54:25 people tend to jump to try to do something fancy

54:29 without first understanding what is the problem

54:31 that we are gonna try to solve.

54:34 And,

54:34 and we,

54:35 we need to have a characterization.

54:38 Of

54:39 of that

54:40 of that problem

54:41 and sometimes going

54:42 through the process of of generating the information

54:46 it's actually

54:47 it's actually interesting

54:49 and what we may find from one context to another

54:52 is different

54:54 so so let me give you an an an actual example

54:57 um

54:58 two different phases in my career probably during my whole career

55:02 I have been concerned about the issue of how to collect taxes right?

55:05 So in my la

55:07 right now is it is an important part of the governance mandate.

55:11 Before that was how to collect federal taxes in,

55:14 in,

55:14 in Mexico as finance minister then.

55:18 In my first incarnation here at the bank

55:20 but before that also,

55:23 also

55:24 when I was finance secretary of Mexico City many years ago

55:27 and uh

55:28 and

55:29 and we have

55:30 and we and we face.

55:32 Some serious problems at the time that was

55:34 almost 20 years ago how about how to collect

55:37 three different sets of taxes property taxes,

55:40 water charges,

55:43 and car property uh uh uh and car property taxes

55:47 and

55:48 through the time I I have been trying to replicate some,

55:51 some of,

55:51 of these studies,

55:53 um.

55:55 Uh,

55:55 but let me start with one example which is what,

55:58 what made me think that

56:00 we have to be careful

56:02 if it's just about providing,

56:04 uh,

56:05 public servants,

56:06 how was your work to,

56:07 to be whatever they could,

56:09 all the,

56:10 all the best they could,

56:11 they could be,

56:12 so,

56:13 so.

56:14 At the time in Mexico City,

56:16 the same taxpayer have a different response

56:19 to which taxes they were willing to pay

56:22 for whatever reason

56:24 they would pay the property tax at a higher rate

56:27 than the water charges,

56:28 and the last one was the car property taxes,

56:31 right?

56:31 And that became really

56:33 really uh uh a problem for me

56:37 and I start thinking trying

56:38 trying to get some basic data actually.

56:41 Uh,

56:41 which was not,

56:42 not easily available.

56:43 Why is people not paying taxes?

56:45 And I,

56:45 and I was getting desperate,

56:47 uh,

56:47 and by the way,

56:48 that's,

56:48 I'm gonna bring me for a quick comment on,

56:52 on,

56:53 on your issue of

56:54 centralized accountability.

56:56 And,

56:56 and I think there's that part of the

56:58 problem that you may find with a centralized accountability

57:01 is that they are asked,

57:02 uh,

57:02 when you have someone which is very powerful and asked

57:05 to solve a problem

57:07 that's not sufficient if they don't describe properly

57:09 the problem.

57:11 So,

57:11 so I may have been partially in that in that in that seat and I was,

57:15 we need to collect this this tax we need to collect the tax,

57:18 but I couldn't understand why it was a problem why people were not collecting the tax

57:22 and I was in a meeting with people who

57:25 who all all all of them

57:27 were were responsible for that,

57:28 right?

57:29 One were uh responsible or actually the database another were

57:33 to to do the

57:34 uh uh the the the the legal claims,

57:37 etc.

57:39 And then it occurred to me something which was not very systematic,

57:43 but I thought it would help me to to understand the problem.

57:46 Uh,

57:47 all of them,

57:48 the public officials sitting on that meeting had their cars

57:51 in the parking lot,

57:53 right?

57:53 So I asked someone,

57:54 why don't you go down

57:56 and check if they are paying the tax.

57:59 Right,

57:59 the ones who were collecting the taxes,

58:01 so somebody won,

58:02 it was not a random experiment,

58:04 right?

58:04 Like it was

58:05 so they come and say,

58:07 well,

58:07 actually they are not paying their taxes,

58:09 right?

58:10 So,

58:12 and

58:12 so part of the problem is that people who were in charge,

58:15 the senior people who were in charge of collecting the taxes,

58:19 didn't themselves believe.

58:21 That they have to pay it

58:23 or that they or the or or or so so

58:27 so it's not always about

58:29 how to

58:30 to to make them the best that they could

58:33 uh that they could be you need to you need to understand

58:37 if they are actually uh

58:39 uh.

58:41 Let me give you another example now with the property with the property taxes um

58:46 because the process of generating the information opens different questions.

58:51 So,

58:51 um,

58:52 so when I tried to revisit that those kinds of issues when I was back at the bank

58:57 around 7-8 years ago,

59:00 um,

59:01 uh,

59:01 I'll,

59:01 I'll try to figure out now why people were not paying the property tax,

59:05 and,

59:05 uh,

59:05 at that time there was already already literature,

59:08 particularly from the UK.

59:09 That say that if people knew that their neighbors were paying the taxes

59:14 there was a a higher propensity for them to pay the taxes so so that the

59:19 the

59:20 underlying hypothesis is that um

59:23 uh there was some sort of uh

59:26 of social uh commitment.

59:28 And,

59:29 and that that by appealing to that

59:31 people will fulfill so rather than

59:33 sending them a letter saying

59:35 you,

59:36 you need to pay your taxes so we're gonna

59:38 go and collect it,

59:40 you will send a letter that say

59:42 80% of your neighbors already pay the taxes.

59:44 Why don't you do it,

59:45 right?

59:46 And so,

59:47 so,

59:47 so,

59:48 so I was replicating an

59:49 an actual

59:50 experiment that I did many years ago.

59:53 And uh when we were sending letters and we we we did it this way right?

59:57 We divided

59:58 um

1:00:00 the letters uh

1:00:01 by

1:00:02 one letter

1:00:03 was written in a very harsh

1:00:07 tone right?

1:00:07 We have detected that you failed to pay your taxes

1:00:11 we are gonna give you

1:00:12 15 days

1:00:13 to pay it.

1:00:14 And the tone and in Spanish,

1:00:16 you know that uh you could address somebody

1:00:18 in a formal way that's sed

1:00:21 or,

1:00:21 or,

1:00:21 or in a friendly way too.

1:00:23 So that was in the user in the very formal way,

1:00:26 right?

1:00:27 Then we we prepared another set of letters,

1:00:30 which was.

1:00:31 Addressing them in the two-way

1:00:34 and say,

1:00:35 you know,

1:00:36 but the property tax to property tax we collect x amount and that serves to pay

1:00:42 the subway system,

1:00:43 the firefighters,

1:00:44 etc.

1:00:45 right?

1:00:46 Trying to appeal to what what cut services.

1:00:50 Could the public perceive as good,

1:00:52 not the police,

1:00:53 because sometimes if the police comes and smack someone.

1:00:56 So

1:00:56 there was another,

1:00:57 another subway line who have a problem that we didn't put that subway line,

1:01:01 right?

1:01:01 Just think that people that people thought it would be

1:01:04 and then we have a control.

1:01:05 The control letter

1:01:07 was a letter that they will send,

1:01:10 uh,

1:01:11 uh,

1:01:11 in a standard way.

1:01:12 And I'm following a very formalistic uh uh a formalistic approach

1:01:17 that letter

1:01:18 uh was starting with a legal basis right according to

1:01:22 the code that we are in charge of collecting these taxes

1:01:25 and then it will say all the places that it could could be paid

1:01:29 so.

1:01:29 You could pay you could be paid in all these banks it could be paid in Walmart.

1:01:33 It could be better

1:01:33 so it has a lot of information so to fit that in that

1:01:37 one letter

1:01:38 they had to use

1:01:39 a very small font,

1:01:41 right?

1:01:42 So it was impossible

1:01:44 to read that letter,

1:01:46 right?

1:01:47 So we send them

1:01:48 right?

1:01:49 and and then one of the letters also we may have a different division

1:01:53 one was sent by the by the fiscal attorney which in principle has a

1:01:57 a higher a higher authority.

1:02:00 So

1:02:01 the result was the opposite of the UK.

1:02:04 So the harshest the letter.

1:02:07 The more collection,

1:02:08 right?

1:02:09 Actually,

1:02:10 the letter that is impossible to read,

1:02:12 collect more

1:02:13 than the nice letter,

1:02:15 right?

1:02:16 And

1:02:17 so,

1:02:17 so that gives you an idea about what should be.

1:02:21 The optimal

1:02:23 Tax collection

1:02:24 uh approach which is be as hard as you could be,

1:02:28 right?

1:02:29 But for me also raised a different question

1:02:32 which I haven't been able to tackle right?

1:02:35 which is

1:02:35 why is this happening?

1:02:37 And,

1:02:38 and,

1:02:38 and in my mind there were two options.

1:02:40 Uh,

1:02:41 one is,

1:02:42 uh,

1:02:43 particularly 20-30 years ago,

1:02:46 uh,

1:02:46 a lot of the revenues from Mexico were coming from were all oil related,

1:02:51 so that this idea that you pay taxes and get some services was broken.

1:02:56 We were getting services without paying taxes

1:02:59 and that's why probably

1:03:00 the people who were actually in my team many years ago

1:03:04 didn't feel that they have to

1:03:05 pay taxes

1:03:06 somehow the money was coming

1:03:09 back,

1:03:09 right?

1:03:10 But that's a different problem

1:03:12 than than understanding why people were paying taxes or no it's

1:03:16 understanding what what was their role

1:03:18 in the social contract.

1:03:21 And that was

1:03:22 one hypothesis the other hypothesis is.

1:03:26 Well maybe the society have a very.

1:03:32 A very stre

1:03:34 stressed relationship with authorities.

1:03:37 They are not willing to cooperate in anything,

1:03:39 only if they are forced,

1:03:41 and that's probably.

1:03:43 And even

1:03:44 much more serious

1:03:47 problem right?

1:03:48 so that's what I thought uh.

1:03:51 If I had time at the time

1:03:53 those are the things that I would have wished to investigate

1:03:56 so but uh

1:03:57 but I wasn't trying to find that those problems though I stumbled.

1:04:02 Into that into that by just trying to figure out

1:04:05 uh how

1:04:07 what were the the general dynamic

1:04:08 so,

1:04:09 so the point I'm trying to say is

1:04:11 some of the questions and some of the relevant questions and even some of the

1:04:15 research questions

1:04:17 are,

1:04:17 are popping up

1:04:19 just by uh by trying

1:04:22 to gather uh uh

1:04:25 uh to gather

1:04:27 information.

1:04:28 Uh,

1:04:28 let me just make 11 more,

1:04:32 uh,

1:04:32 one more comment,

1:04:34 um.

1:04:35 Uh,

1:04:36 which is,

1:04:36 um,

1:04:39 So I really like your idea of the innovation.

1:04:43 I have all,

1:04:44 and I think it makes,

1:04:45 it makes,

1:04:47 it makes a difference.

1:04:48 I'm only,

1:04:50 I have always been worried.

1:04:52 To how,

1:04:55 how,

1:04:55 if it's possible

1:04:57 within a government and a bureaucracy

1:04:59 to move to a permanent innovation

1:05:01 mode

1:05:03 actually bureaucracies are about the opposite

1:05:06 bureaucracies are about finding a way to do something

1:05:09 and just keep doing it.

1:05:11 That's the caricature of bureaucracy.

1:05:12 They,

1:05:13 they,

1:05:13 they refuse to do it in a different way.

1:05:15 Right.

1:05:16 And as some someone told me once,

1:05:18 this is bad and good.

1:05:19 Also,

1:05:20 when bad governments come with what they keep doing what they are supposed to,

1:05:25 uh,

1:05:25 they are supposed to do it.

1:05:27 So,

1:05:27 so this innovation could be good and disruptive

1:05:30 when you change that process,

1:05:32 but then it has to be codified

1:05:34 in a.

1:05:35 Uh,

1:05:36 in,

1:05:37 in a,

1:05:38 in a process,

1:05:39 and bureaucracies are good about it.

1:05:40 When,

1:05:41 when I first was a general director in Mexico,

1:05:43 they sent a team to try to understand what we were doing in each process,

1:05:47 and they,

1:05:47 and they,

1:05:48 they start going

1:05:49 desk by desk,

1:05:50 right?

1:05:51 And at some point they call me and say,

1:05:53 we found something which is.

1:05:56 Which is a little bit of concern.

1:05:58 These guys reviewing some papers,

1:06:01 check them,

1:06:02 classifying two different parts,

1:06:03 put a seal to one and say,

1:06:06 And they don't use that thing for anything for the last 15 years,

1:06:10 right?

1:06:11 So some somebody forgot to tell

1:06:13 that person that

1:06:15 that thing that he has been doing.

1:06:16 So,

1:06:17 and,

1:06:17 and in,

1:06:18 in,

1:06:18 and actually they say,

1:06:20 please don't let him know.

1:06:22 Yeah,

1:06:22 it's,

1:06:23 it's gonna,

1:06:23 it's gonna be,

1:06:24 it's gonna be traumatic,

1:06:25 right?

1:06:26 So,

1:06:26 but on the one sense that's an example of

1:06:30 if you codify it,

1:06:31 people just keep doing it.

1:06:34 Right,

1:06:34 but you have to ask

1:06:35 why they are doing it,

1:06:37 so let me stop here.

1:06:39 Thanks Actor,

1:06:40 and,

1:06:40 and stretching the James Bond analogy,

1:06:42 um,

1:06:43 and

1:06:44 to go on that point,

1:06:45 which is

1:06:46 the number of movies where he goes rogue

1:06:49 is is a fair share of them.

1:06:51 So working in a bureaucracy in order to get things done,

1:06:53 do you have to go rogue?

1:06:55 Anyway,

1:06:56 um,

1:06:56 I have a few questions in the chat,

1:06:58 but let's me just take a sense of the room if people have questions or comments.

1:07:02 Um,

1:07:03 OK,

1:07:03 let me take these two,

1:07:05 starting here and then there.

1:07:08 OK,

1:07:09 so from a,

1:07:10 I'm a very operational person.

1:07:12 I'm the implementation guy.

1:07:14 So from your talk,

1:07:15 I gathered that may be 3 areas of implementation

1:07:17 that we can incorporate in our project designs.

1:07:21 One is the implementation of some kind of performance management systems.

1:07:25 The second is some kind of analytics,

1:07:27 implementation of analytics,

1:07:28 data analytics.

1:07:30 And the third is some additional training

1:07:34 to the civil servants,

1:07:35 training on diagnostics,

1:07:36 training on innovation.

1:07:38 So this is very positive and it looks like

1:07:41 it has impact,

1:07:43 but on the other hand,

1:07:44 we have a large number of failed civil service reform projects in the bank.

1:07:49 So how can you integrate your findings

1:07:52 with the lessons learned from our past failures?

1:07:56 Thanks.

1:07:59 Thanks.

1:08:00 I have a question about training

1:08:02 innovators or creating innovators.

1:08:05 Uh,

1:08:07 do you have any insight into what,

1:08:09 what does it take to make an innovator?

1:08:11 What's the recipe?

1:08:12 Like,

1:08:12 is it,

1:08:13 you know,

1:08:13 is it,

1:08:14 do you provide training on administrative skills,

1:08:16 some part,

1:08:17 intrinsic motivation,

1:08:20 etc.

1:08:21 Um,

1:08:21 and secondly,

1:08:22 along those lines from your data,

1:08:25 Is there a cutoff point

1:08:28 for competency among

1:08:30 civil servants and bureaucracies?

1:08:32 I'm thinking like,

1:08:33 I'm working in Papua New Guinea in a very capacity,

1:08:36 capacity constrained environment.

1:08:38 So,

1:08:39 when I work particularly down at the level of a school or head teacher or a district,

1:08:45 right?

1:08:46 They often lack

1:08:48 The basic competencies to fulfill their roles,

1:08:51 whether it's the,

1:08:52 the,

1:08:52 the training or what,

1:08:54 uh,

1:08:54 or the diploma.

1:08:56 Um,

1:08:57 so,

1:08:57 like,

1:08:58 at what point

1:08:59 do you,

1:08:59 does a public servant just need to have the capacity,

1:09:02 uh,

1:09:03 and,

1:09:03 and the competency to do their job?

1:09:05 And at what point do you train them to be an innovator?

1:09:08 Like,

1:09:09 where's the cutoff?

1:09:12 So let's turn it back to you,

1:09:13 Dan,

1:09:13 if you want to sort of react to Arturo and then

1:09:15 see what you can react to those questions.

1:09:17 Yeah,

1:09:18 very quickly,

1:09:18 I mean,

1:09:19 Arturo.

1:09:21 Thank you very much for being here for discussing and for your ongoing support.

1:09:24 I think,

1:09:25 as you say,

1:09:25 this is something the practice is doing at a

1:09:28 a more vibrant scale than ever before,

1:09:30 and I think I take my hat off to everyone involved in that.

1:09:33 You know,

1:09:34 in terms of

1:09:34 the idea that we're freeing public servants

1:09:37 is not what I'm getting at,

1:09:38 is that we're giving structure.

1:09:39 You saw structure is what people want,

1:09:41 but I think we have very little sense

1:09:43 of what that structure should look like.

1:09:45 We've been taking pro forma ideas for a

1:09:47 long time without really experimenting with them or,

1:09:49 you know,

1:09:49 utilizing,

1:09:51 uh,

1:09:51 even measurement of their long-term effects.

1:09:54 And so

1:09:55 here,

1:09:55 an appraisal reform that was sold

1:09:57 to the service was more effective,

1:09:59 you know,

1:10:00 than one,

1:10:00 where you're just basically leaving it to centralized actors to impose it.

1:10:04 That's just one example.

1:10:05 So structure's important,

1:10:06 and I think that's also true of

1:10:08 your,

1:10:08 your question around,

1:10:10 you know,

1:10:10 should we

1:10:12 take the conservatism of government and convert it purely to an innovation.

1:10:15 I think that's the fantastic thing about a public administration because it locks

1:10:19 together actors with very different incentives in

1:10:21 a way that many institutions don't.

1:10:23 And so there's nothing

1:10:25 in,

1:10:26 you know,

1:10:27 an innovation,

1:10:28 uh,

1:10:29 focus that means that everyone has to be an innovator.

1:10:31 There are some aspects of government

1:10:33 that should rightly be conservative,

1:10:35 and yet,

1:10:36 why not have someone arguing with them around the table?

1:10:39 And

1:10:40 over time,

1:10:41 it's having the.

1:10:42 Institutions that effectively mediate that conversation

1:10:45 in government that's going to work.

1:10:46 It's a little bit like DC and the bank.

1:10:48 DEC is supposed to be an innovator,

1:10:50 creating new things.

1:10:51 And so we have a machinery that allows some of DEC's

1:10:54 work to influence and others that are more like a laboratory.

1:10:58 And so I think it's the machinery that sits

1:10:59 above those that's going to mediate its success.

1:11:03 You know,

1:11:03 um,

1:11:05 what you were saying about surprises,

1:11:06 how many surprises you've had in public policy is exactly what I found.

1:11:10 This is a very different place to

1:11:12 the worlds that we have

1:11:14 been able to gather information on before.

1:11:16 And so I would say that

1:11:18 pushes us to be more innovative because at the

1:11:20 moment we're probably doing things that aren't that effective,

1:11:23 and we're doing them systematically.

1:11:25 I would say we need to systematize the innovation.

1:11:29 It goes a little bit to your idea that civil service reforms have failed.

1:11:32 I don't know where that idea came from.

1:11:35 Do you,

1:11:35 you know,

1:11:35 I don't think we have very good evaluations of civil service reforms,

1:11:38 you know,

1:11:38 there's obviously

1:11:39 classic literature on the IEG report that sort of said these things don't work.

1:11:44 If you talk to the IEG editors,

1:11:46 they actually said that's not what we were saying.

1:11:48 And we've had that,

1:11:50 and I have met with them,

1:11:50 and what they said was we need better diagnostics to identify what works where.

1:11:55 So I pushed back on the idea that civil service reform doesn't work.

1:11:57 Maybe writ large,

1:11:58 trying to get the entire service to completely transform.

1:12:02 Why would we need to do that?

1:12:03 We can move agency by agency.

1:12:06 The.

1:12:07 Many small changes that come from the Federal Viewpoint Survey in the United States

1:12:12 make for a very large change in the way that the

1:12:15 public service works in the US,

1:12:17 but it's individually based.

1:12:18 It's,

1:12:18 it's one manager changing one thing at a time.

1:12:21 But it's a technology that's been made available

1:12:23 to all managers that's making the big change.

1:12:26 So,

1:12:27 you know,

1:12:27 I,

1:12:27 I would say we need to,

1:12:29 to push back on the idea.

1:12:30 I don't think we know that civil service reform.

1:12:33 In its flavors that were say in the IG report

1:12:36 didn't work.

1:12:36 We just don't have the data.

1:12:38 And then going back to the sort of the idea of training and,

1:12:41 you know,

1:12:41 to what extent do these things that have surprised me,

1:12:44 this is very fresh.

1:12:46 I mean,

1:12:46 I'm now doing 3 other IEs all around in training because I'm like,

1:12:51 We don't know,

1:12:52 we better go and start finding stuff out.

1:12:54 One of those IEs has films of Ethiopian public servants being

1:12:59 James Bond characters,

1:13:00 not quite,

1:13:00 but you know,

1:13:01 being heroes and and get making souls.

1:13:03 Is it a narrative that we need to change?

1:13:05 Other IEs are much more structural.

1:13:07 And so,

1:13:08 you know,

1:13:08 I can give you guesses at the answers to those questions.

1:13:11 One is,

1:13:12 you know,

1:13:13 capability is,

1:13:14 I think,

1:13:14 something that

1:13:15 is kind of imposed by us

1:13:17 because

1:13:18 my kids

1:13:19 are very capable of changing things that they want to change.

1:13:23 As long as you're asking the public servants what they think are the problems

1:13:27 and helping them,

1:13:28 and that Ghana,

1:13:29 we didn't say this is the process you need to change.

1:13:31 What they said was a lot of de jure

1:13:34 processes have been put into Ghana's public service.

1:13:36 We just don't do them.

1:13:38 And so I think 70% of all of the change innovators was to move the de facto

1:13:45 to the de jure.

1:13:46 And I would say that's going to be true across most of our client countries.

1:13:50 There's very nice de jure rules,

1:13:52 what you're supposed to do,

1:13:54 and yet most people do not follow them.

1:13:56 And so what the public servants wanted to do

1:13:58 was basically shift their departments towards the de jure.

1:14:03 So I think if you talk to your teachers,

1:14:05 you talk to your education officers and the local governments and say,

1:14:08 what do you think's the big bottleneck,

1:14:10 let me give you some tools

1:14:12 that any level of capability can use to solve that problem,

1:14:16 I think that's the way we think about this,

1:14:18 not

1:14:19 your school should be teaching

1:14:20 with this curriculum,

1:14:21 let me help you get there.

1:14:24 So I can make more comments,

1:14:25 but I'd love to hear if there's a couple more questions here.

1:14:28 So let me call on Juan.

1:14:30 Juan,

1:14:30 are you still online?

1:14:31 Juan Marron?

1:14:33 You had a question in the chat.

1:14:34 Do you wanna come on screen and ask your question and make a comment?

1:14:37 Can

1:14:38 you hear me?

1:14:38 Go ahead.

1:14:38 Yes,

1:14:38 we can hear you.

1:14:40 Yeah,

1:14:40 no,

1:14:41 thank you,

1:14:41 Dan.

1:14:41 It,

1:14:41 it was a

1:14:42 thing.

1:14:43 I,

1:14:43 I think that

1:14:44 a little bit of a call to the,

1:14:46 to the governance,

1:14:47 um,

1:14:48 global practice and education,

1:14:49 global practice in education you have heard Jaime

1:14:52 talking about a lot about implementation capacity.

1:14:55 Uh,

1:14:55 and it has been a struggle,

1:14:56 right?

1:14:57 So,

1:14:57 so we did like an analysis of how many different,

1:15:00 um,

1:15:01 education,

1:15:02 uh,

1:15:02 multilaterals,

1:15:03 bilaterals do

1:15:04 the training and it's a complete failure of what we have been taking as,

1:15:08 as,

1:15:09 as a success.

1:15:10 And,

1:15:10 and a lot of,

1:15:12 of that is what you were saying is that we bring like public officials training on,

1:15:16 on something but honestly I've been in

1:15:19 Very different types of countries and

1:15:21 that is just like a way to get per diems,

1:15:24 right?

1:15:24 So,

1:15:24 so it,

1:15:25 it is,

1:15:26 that is the reality.

1:15:28 Um,

1:15:28 so I think one place where,

1:15:30 where we could work together,

1:15:32 um,

1:15:33 uh,

1:15:33 from the research side,

1:15:34 from the operation side is,

1:15:36 is,

1:15:36 is how to measure capacity but in a way that is a little bit faster.

1:15:40 We don't have the time,

1:15:41 we don't have the,

1:15:43 the expertise and,

1:15:44 and,

1:15:45 and,

1:15:45 and the needs are there to just

1:15:47 get some analytics on,

1:15:48 on,

1:15:48 on capacity.

1:15:49 Capacity for what,

1:15:50 for example,

1:15:50 is the first thing that

1:15:52 that That,

1:15:52 that I asked.

1:15:53 So that's one.

1:15:54 The second one is I really like Arturo's point on what the problem is,

1:15:57 what is the problem,

1:15:58 right?

1:15:58 So that we're trying to solve,

1:16:00 but

1:16:00 I,

1:16:00 I see many countries getting in so,

1:16:02 so much analytics,

1:16:03 but then they don't know what to do with the data.

1:16:05 So why are we collecting this data for,

1:16:07 right?

1:16:08 So I think,

1:16:09 I think your call for analytics,

1:16:10 uh,

1:16:11 uh,

1:16:11 matches and aligns very well with what Arturo was saying about,

1:16:15 um,

1:16:15 um,

1:16:17 uh,

1:16:17 innovation.

1:16:19 Um,

1:16:20 and then the other thing that struck me a lot that,

1:16:22 that I think that I feel that as I advance in my

1:16:24 career at the bank that I feel more frustrated with and,

1:16:27 and have voiced here

1:16:29 is this division among people who are

1:16:31 doing impact evaluations and research and operations.

1:16:34 I was very saddened by seeing like

1:16:36 how your recommendations are different from these,

1:16:39 when,

1:16:39 when,

1:16:39 when actually you showed up very Nice presentation

1:16:42 saying it's like we don't work well together,

1:16:44 right?

1:16:45 So,

1:16:45 so that,

1:16:46 that's what we show in the

1:16:47 analytics and this is a call for all of us.

1:16:50 It's like how do we start bringing you uh

1:16:53 this impact evaluations as part of our preparation,

1:16:55 as part of our,

1:16:57 our,

1:16:58 um our project preparations.

1:16:59 I think that is where we can really have like like impact.

1:17:02 I'm not saying we shouldn't do

1:17:04 A very good impact evaluation,

1:17:06 but we,

1:17:06 we need to bring a lot of the,

1:17:08 of the insights of all these research into our operations.

1:17:11 Otherwise we will continue having great,

1:17:14 great analytics,

1:17:15 great research that honestly many of us don't use.

1:17:19 So,

1:17:20 so not to be critical,

1:17:21 just a call to work together on,

1:17:23 on bringing operations and research a little bit more closer.

1:17:26 Thank you.

1:17:28 Thanks,

1:17:28 Juan.

1:17:28 And before we start closing up,

1:17:29 um,

1:17:30 uh,

1:17:30 there's a question by Charlie.

1:17:32 Charlie,

1:17:32 do you wanna come in?

1:17:35 It just says Charlie,

1:17:36 so

1:17:37 I have no context for that name.

1:17:41 It's about to end and uh thanks for a

1:17:43 fantastic invitation.

1:17:45 Uh,

1:17:46 we can't hear you very well.

1:17:48 It's cutting in and out,

1:17:49 um.

1:17:51 across the street with the computer

1:17:55 on the screen.

1:17:57 Sorry,

1:17:57 Charlie,

1:17:57 we,

1:17:58 we,

1:17:58 there's something amiss with your mic,

1:18:00 it seems.

1:18:05 I'll give you a minute.

1:18:06 Otherwise I could read your question out from the chat,

1:18:08 but

1:18:08 I'd rather you,

1:18:09 if,

1:18:10 if you have another mic.

1:18:14 The.

1:18:18 Then

1:18:20 What's that?

1:18:28 Try to switch my mic.

1:18:31 You hear me?

1:18:33 Very quietly.

1:18:36 OK,

1:18:39 Maybe I'll just,

1:18:40 I'll just read out the main points from your question if that's OK.

1:18:43 Um.

1:18:45 So,

1:18:48 The question,

1:18:49 the emphasis is that there's a way to foster individual innovation leaders can

1:18:52 be seen to be counter to other

1:18:53 things that characterize better performing bureaucracies,

1:18:56 sprit de corps,

1:18:57 the peer pressure

1:18:58 and support.

1:19:00 How do you square those two issues,

1:19:02 um.

1:19:03 He says we don't really want rogue bonds.

1:19:06 Um,

1:19:07 second question is that the Pakistan example of individual skills would,

1:19:11 would presumably have so many exogenous variables that

1:19:13 are more determining than individual management characteristics.

1:19:17 Even if one can control for the individual,

1:19:19 would that same person really be as effective in another

1:19:22 part of the country with another set of variables?

1:19:25 And then he has an operational question,

1:19:27 which I think I'm actually also curious about,

1:19:29 which is much of what you suggest would fit the old

1:19:31 Lil type of operations,

1:19:32 and for those of you under 40,

1:19:34 that's learning and innovation loans.

1:19:37 Um,

1:19:38 which

1:19:39 are now defunct.

1:19:40 Our operational management incentives and our legal framework are

1:19:42 not well geared to projects with lots of adaptation,

1:19:45 which is something I know we all

1:19:47 struggle with.

1:19:48 Um,

1:19:48 OPCS would benefit from this.

1:19:51 OK,

1:19:51 um,

1:19:52 any last questions from the room

1:19:54 before I turn it back to,

1:19:56 to,

1:19:57 I'll give Artu,

1:19:57 I'll give you the last word,

1:19:58 OK,

1:19:59 um,

1:19:59 a question here,

1:20:00 and then then and then Arturo.

1:20:04 The.

1:20:06 Just a quick question on how the innovators that were trained

1:20:09 were selected because that seems super important and I think I,

1:20:12 I missed that

1:20:14 for the treatment that worked.

1:20:18 Wonderful.

1:20:19 Um,

1:20:21 I think the idea that we have innovators,

1:20:24 rogue,

1:20:26 going rogue is worrying to everyone.

1:20:29 I think an a successful innovator,

1:20:31 the way it was sold here was very much that you're going back

1:20:35 to be part of a unit or division that you are going to change.

1:20:39 And so therefore,

1:20:40 innovation and innovator here,

1:20:42 in fact,

1:20:42 is

1:20:43 binding the individual back to the service that they're trying to change.

1:20:47 So in some sense,

1:20:48 you could imagine James Bond.

1:20:50 Going back to MI6 and reorganizing it would actually not have been going rogue but

1:20:54 bringing really good consultancy that would have

1:20:55 been useful for the rest of the service

1:20:58 versus going off and

1:21:00 on a vendetta that he was worried about.

1:21:02 And I think we're talking,

1:21:03 especially in the Ghana case,

1:21:04 but I think more generally,

1:21:05 we're talking about that one.

1:21:08 The trainees

1:21:09 are those that were sent to.

1:21:11 The

1:21:12 standard public service training,

1:21:15 this is a kind of rotated part of every year,

1:21:17 so,

1:21:18 um,

1:21:18 the random,

1:21:19 randomization happened as you hit the training institute.

1:21:23 And so you're

1:21:25 comparing,

1:21:25 this is sort of external,

1:21:27 these are folks who would have come to training.

1:21:29 These are the folks who would have gone through their annual or

1:21:32 in our case,

1:21:32 2 years' worth of training.

1:21:35 Um,

1:21:36 but I think it comes back to this idea of,

1:21:38 you know,

1:21:39 how do we embed innovation in our operations?

1:21:42 And I would say that

1:21:44 right upfront,

1:21:45 you know,

1:21:45 a lot of operations are not taking the evidence that already exists.

1:21:49 And so I think just starting every operation,

1:21:51 sitting down with a load of deck colleagues and trying

1:21:54 to understand what evidence and data we can already apply.

1:21:58 Secondly,

1:21:58 is then saying,

1:21:59 well,

1:21:59 what's useful for us to know for the next operation in 4 years

1:22:03 and starting the impact evaluations then,

1:22:05 and absolutely,

1:22:06 Juan,

1:22:06 you know,

1:22:07 a lot of what we've done,

1:22:08 so for example,

1:22:09 in Ethiopia,

1:22:10 is to do short IEs.

1:22:11 So we're doing a report for the

1:22:14 Ethiopian Management Institute now.

1:22:16 While the longer term evaluation goes on,

1:22:18 that's how we did in Tanzania.

1:22:20 And so I think in some sense,

1:22:22 what you should expect from your colleagues

1:22:24 in DC is this portfolio ability that we're

1:22:28 using a whole range of tools to build the knowledge that we need,

1:22:31 then sitting down with the colleagues before they

1:22:34 start the education program in 3 or 4 years

1:22:36 and bringing that knowledge back in.

1:22:38 So it's using our project today as a platform for doing better projects tomorrow.

1:22:46 Thanks Dan.

1:22:47 So Arturo,

1:22:48 last word to you.

1:22:49 So,

1:22:49 so,

1:22:49 so very quickly,

1:22:50 first of all,

1:22:51 uh,

1:22:52 analytics basic basic data,

1:22:54 that's super important,

1:22:55 right?

1:22:56 And it could come from different ways.

1:22:57 It could come from just

1:22:59 computing

1:23:00 raw data,

1:23:01 administrative data,

1:23:02 which is already there from getting insights from,

1:23:05 from,

1:23:05 from,

1:23:06 uh,

1:23:06 from studies

1:23:08 then.

1:23:10 Having the right

1:23:11 definition of the problem that we want to address

1:23:16 that's critical

1:23:17 that that that's not that's that's not negotiable

1:23:20 and let let me just give you one example many years ago I was manager here.

1:23:24 And there was uh

1:23:25 this uh

1:23:26 this uh grants that the one that the,

1:23:29 the,

1:23:30 the bank was gonna guard to,

1:23:31 to junior staff

1:23:33 who come with an innovative

1:23:35 uh way to use telephones to solve a problem and then I got one proposal from one staff.

1:23:41 And I keep asking her,

1:23:42 what's the problem you try to solve?

1:23:44 And at the end,

1:23:45 it was clear that she just wanted to use a smartphone to do something

1:23:48 and she was shifting the problem as I was questioning.

1:23:52 So shouldn't be

1:23:53 about using phones

1:23:54 or an RTC or behavioral economics.

1:23:58 first

1:24:01 you need to define the problem first and then you

1:24:04 figure out which is the which is the right answer.

1:24:09 Great,

1:24:09 thanks.

1:24:09 I think we've thrown out a lot of challenges here,

1:24:11 both,

1:24:11 um,

1:24:12 sort of in our engagements,

1:24:13 in our internal,

1:24:14 uh,

1:24:15 organization,

1:24:16 um,

1:24:17 and I thank both of presenters

1:24:19 discussing and thank all of you for,

1:24:21 for coming in the room and thanks for you,

1:24:22 uh,

1:24:23 online as well for,

1:24:24 for engaging in the questions.

1:24:26 And with that,

1:24:26 let me call it to an end and just join me in thanking the presenters.

showAllTimestamps

no

transcript

Well, welcome everybody. We're very excited today to, to welcome you, uh, here to this May edition of our policy research, research talk series. Uh, this time presented to you jointly by, uh, the Development Economics vice presidency's Research Department, as well as DIM, the Impact evaluation department. As you may know, these talks provide us a chance to share the research coming out of uh the World Bank with the goal of sharing the findings with colleagues inside the institution as well as outside. Um, I'd like to welcome our audience both on Webex and YouTube, as well as those of you who made it into the room. Thank you for coming. Um, today I'm pleased to introduce my colleague Dan Roger, who is a senior economist in and Program manager for DM's governance and Institutions building team, as well as the co-founder of the World Bank's Bureaucracy lab. Uh, Dan will be following up on his very entertaining and exciting and notorious 2019 policy research talk, um, which has a similar starting point to today's, I think, uh, you'll see, uh, where we go, I'm going with that. Um, more data are available on public officials and the behavior of public organizations than ever before, and Dan has not only been leading the effort to collect these data through the bureaucracy lab. But also helping to make sense of it through his own research. So picking up where he left off his previous talk here with us at the policy research talk series, Dan will look into new evidence on the design of accountability structures, the role of public service culture, and methods for better understanding public administration. And just, you know, Dan's been with us for a while at the World Bank, but just to flag that before he worked at the World Bank, he was an economist in the presidency of Nigeria and as an associate researcher for UK's Department for International Development. So a little bit of broad bureaucratic experience there. Uh, additionally, we're extremely grateful today to have Arturo Herrera Gutierrez as our discussant. Arturo is the Global Director for the Governance Global Practice in the Equitable Growth Finance and Institutions Practice Group. A vice presidency, uh, he put that puts him in a unique position to discuss, uh, Dan's body of research and what he'll be describing today. Arturo's experience, uh, extensive experience in governance from both a practitioner and academic perspective. Uh, within the bank, he served in numerous positions, and I don't even know if I have all of them here, but I have sector, senior public sector management specialist and sector manager in the Latin American Caribbean region. Uh, practice manager in various units include, uh, for global governance, practice in LAC, East Asia Pacific, as well as the global unit. Um, between 2018 and 2021, he left the bank and held leadership positions in the government of Mexico as co-head of the finance team in the presidential transition. Uh, team, uh, Deputy Finance Minister, and most recently, before rejoining us, uh, back at the bank as Minister of Finance and Public Credit. So I'll ask Dan to talk for about 45 minutes, uh, and he may need to walk slowly, after which we'll hear from Arturo for, for 10 to 15 minutes, uh, and we'll conclude the sessions with Q&A from you in the audience. If you have a question, please use the raised hand, um, option in Webex or signal to me in the chat that you do have a question, and I'll call on you and hopefully you can ask your question yourself. Of course, if you're in the room, just raise your hand. Um, and if you're following on YouTube, please submit your question on the YouTube chat, and that'll get relayed to me. So just a reminder to everybody, we are recording the session, and if you're following online, please mute when you're not speaking. So thank you, and with that, over to you again. Thank you. Thank you very much for attending. No problem. But who's presenting the policy research talk? Rogger. Daniel Roger. I think that's what most of you are coming for, so if you want to leave now, you can. I would like to begin by building on my last policy research talk, which was really a, a review of the literature on bureaucracy, on. How you fix bureaucracy through incentives and and create and I left off on culture. But before I do that, I wanna take a step back and think about the focus of these talks. I want to. Think about how we influence policy. And I want to present to you a thought experiment, which I call a kind of model of policy influence that summarizes the way that I think about policy influence and present what I think recent evidence implies for that model and hopefully provide you some motivation for why we should think about this subject today. So let's think of policy as frontline decisions, decision points that public officials have to make. And these decision points are typically embedded inside a program, a program of activity in public service. These programs inside sectors like health and education, and those inside government administration. And so for me, when I think about public policy, I think of the scale of public policy, the scale of these decision points as relative to the scale of government. So how big is public policy? How big is government? Well, there are about 500,000 local governments in the world. Something like 183 national, various regional, like 2000 state and province. So now let's think. In each of these governments, there's something like 10 sectors, health, education, water, and so forth. And back of the envelope, very rough approximation. In each of those sectors, there's probably 10 programs. You can think of them, 10 budget line items. And for each of those programs, there's 10 decision points. Which village should I operate in? What kind of technology should I use to drill a borehole? What timing should I have? Just summing all this up means that every year in the budget, there's something like half a billion budget decisions per year. That's the policy we're trying to influence. This is a rather daunting number to me. I don't know how many public policy decisions you've influenced in your life, but to me this is just an amazing amount of public policy to influence. And of course some work influences multiple decisions at one time. But given the need for validity of any research for a particular context or decision point, it's still a very substantial number. So as an empirically founded institution, what we want to know is a little bit about the elasticities of the nodes at different points in this model. Where can you have impact? And so We could invest in decision points, we could spend our time trying to work out which way we should go on a specific decision. Or we could say, well, we want to take a holistic approach to a decision point and understand how to operate a program. And then of course one could think, well, let's invest in a sector. Work out how to get health going, but as soon as you start working with an entire sector, you're into issues of administration, of how to run the personnel and the organization of these programs. And so you're into issues of whole of government. My big question is, how do we affect these different nodes? How much does an investment actually have influence? And so let's start with the classic decision points. This is actually quite hard, even though we all work on how to improve the decisions of public policymakers. What is an average effect that we should expect? At DM, we've tried to assess the portfolio of IEs that we've done, and our sense is that a good aspirational number is something like doubling the effectiveness of a decision point. That's the kind of big number. Now, you know, many of the evaluations don't get something this large, and there are a few outliers, but there's something aspirational about this, and we have been able to do this in quite a number of IEs. In preparation for this talk, I looked at the documents underlying another big assessment of development effectiveness, the Copenhagen Consensus, and that number, the idea of doubling. Is broadly in line with some of the elements of the Copenhagen Consensus. And then if you spend your time looking through the JPARL RCT database, this idea that you can double development effectiveness is not too far from the truth for a body of evidence. So if that's what you can do with a decision point, what about the administration above it? Because in my model, all policy is embedded inside an administration. So I can now show you evidence from a range of settings on that, but I want to start with the graph I showed you at the end of my first policy research talk. These are slides from that talk, and the idea that came across is that bureaucracy is fundamentally diverse. And actually, one of the stylized facts I'd say of public administration empirical work over the last 4 or 5 years is that that is true. That bureaucracy is just a very diverse place. Some things work very well, some things fail. And when you start to think about this, this is ex post kind of obvious. There are few systematic forces driving homogeneity in the public service, and so. If Bureaucracy is fundamentally diverse. If the administrative setting is such, what does that mean for influencing policy? Well, Let me take you back to the graph in Ghana that I showed at the end of my last graph. What this shows us is the 35 core public institutions of Ghanaia's civil service. These are the ministries that we've worked with, and they rank our data collection over 3 years of how many tasks that they were supposed to complete that they completed. So the Y axis is the proportion completed, and just to be clear, those underneath the graph, those underneath the dots are completed tasks. Those above are uncompleted tasks. Ghana is in the middle of government effectiveness on the WGI for a lot of the kind of clients that we work with. This is not unusual. But the first thing you think is, wow, that is a lot of uncompleted tasks. And of course, the first thing that I think when I see that is that the average investment in a good decision, a good design of a point is going to be heavily mediated depending on where you are in that distribution, which ministry you're working with. So Even if you're working with any but the very best organization in Ghana, you immediately lose 40% of your development impact if you haven't worked with the administrative environment. Just because you weren't thinking about that parallel infrastructure for planning. And if you work with an organization in the bottom, 75% of performers. You've lost all of the gains in development, uh development finance that arises from better design because you're losing 50% of the tasks that are supposed to be completed. The next graph I showed you. Was not just the organizations, but stacked on the solid dots, which are the ministries are the units inside those, those ministries. What you see is incredible variation within the same organization. So let's take A ministry about in the middle of the distribution. And see the 3 units that are on this dotted line. What you find is that if you're lucky and your program or your decision point, the thing that you've been working on, is on the right side of the corridor, that unit that gets things done, and that's the 0.8%. Over three years, 0.8%, 0 sorry, 0.80% of tasks were were completed. But what if you're unlucky? What if when you mentor that ministry and you turn left, you, the unit you're working with is on the left-hand side of the corridor? Well, you're down at that 40 10% point. You've lost 90% of your development effectiveness because you're working with the unit on the left side of the corridor. And what about the one at the end of the corridor? That over 3 years produced nothing. To me this motivates immensely the idea. That thinking about public policy isn't just about good design of decision points. It's about trying to work with that unit or trying to raise these units further up this dotted line. And as I said, is this a Ghana phenomenon? Absolutely not. Now, this is some data from myself, from my co-authors, from my friends, from people in the community. We now know that this 0 to 100% distribution across ministries, departments and agencies is everywhere, everywhere. And so let me take you one more step along this journey and say, Let me take you to Slovakia, where we've been working on productivity data and we even have data at the individual level. So let me take two tasks. So this task is business licensing, and what you see here is a graph of caseload, the amount of cases that an individual officer was able to throughput in a month. And this is over a multi-year period, so these are pretty good, uh, sense of how productive these individuals are. And once again, those orange tri triangles show you the organization, the district average. Huge amounts of variation across districts, but even more amongst individuals. And this is true across all the different types of tasks that we look at. On the right is environmental regulation. People, officers looking at cases of environmental regulation. So in Slovakia, now it's not just turning to the right unit in the right organization, it's turning to the right person. The public administrator is going to hugely mediate your capacity to convert better decision points into development impact. And so when I think about this model, I think to myself, Well, then surely we should be working on how public administration works, as well as working on the good design of individual projects, individual decision points within those projects. But I also think about the scale. Half a billion decision points a year, how are we going to affect that? Working on individual decision points is going to be tough to influence policy writ large. But if we influence the machine that defines those decision points, maybe we have more of a chance. It was interesting to hear Esther Duflo's reflections two weeks ago at the Dime Impact Week, when she said summarizing her entire body of experience and work. But what matters enormously in generating successful development impacts and knowledge are the details of implementation that no one can think of in advance. And they are solved in partnership with our often government partners. And we also all knew this. Anyone who's ever tried to implement a project or program in the field knows if you get the right field coordinator, manager, individual public official that you're working with, everything goes right. But if you get the wrong one. You never do the evaluation. You never roll out that project. The operation doesn't work. But we know this, and yet it is not at the center of how the bank works, nor at the research that we do. If public administration is so mediating and here's all the data to show it, then surely it should be something we take very seriously. So that's what I hope to do in the slides next. Now, before I start, I have heard many times, Mr. Bond, you are forgetting about politics. All bureaucratic outcomes are surely just defined by the political settlement. So I'm gonna push back on this idea partly reflecting on the model I've just showed. I do believe, as I said in my first policy research talk, that political settlements clearly matter for aspects of bureaucracy and they're part of the development process, but At the scale that I've just shown you, half a billion decision points a year, it feels hugely, implausibly demanding that politicians are influencing every single one of those settlements, every single one of those decision points. Personnel, informational, cognitive, and other resource constraints likely apply to politicians everywhere, and much and make it much more likely that there are areas of passive inefficiency that have real welfare consequences. And this will be true of individual decision points, programs, and whole sectors. And the paper that I think articulates this best is Bandiera Etal looking Italian procurement. They even split the waste they see into passive inefficiency and active inefficiency. And what they find is that 85% of inefficiencies are passive. That is an exciting number. Because it, it means we can go and search for the 85% of inefficiencies in administration, or 60% or 30%. That's a lot of work for us to do. It's still indicative of lots of room for a better administration to make a difference to people's lives. But let me discuss how I think. We should go about doing this, and it takes off from the last words I said in my policy research talk, is that bureaucratic culture fundamentally mediates effectiveness. Now there's fantastic qualitative work coming out on exactly this, and it points to that functional bureaucracies are characterized by strong interpersonal bonds, joint accountability, not to us but to each other. And so the question those of us who work in more quantitative and intervention-based work want to understand is, well, What are the interventions we might actually intervene in service to get government functioning with these time team dynamics? So I'd like to present to you 3 impact evaluations to start that indicate where I think we should go. I'm going to begin by assessing perhaps the most common notion of an effective bureaucratic culture, that of centralized accountability. The idea that a senior member of government, a strong man, and it is almost always a man, can drive public sector performance from the center. This is an often. Talked about idea that this is how you change the culture of the public sector. So my co-authors and I assess the impact of a showcase centralized accountability system implemented in Punjab province in Pakistan. And so the scheme allows us to say something about what happens when you try to drive performance culture from the center using fantastic data. An amazing data system was put into place. And our core interest is whether senior management can sort of force an improved public sector performance, having been empowered by this management information structure. So what we do is we take a 6 year eva we evaluate a 6 year period in which the Chief Minister of Punjab was given detailed data on how each educational district was performing and alerted to the failure of an individual bureaucrat. And then that failure would cause great de facto punishment. We assess the impact at initiation and then on bureaucrats over these six years who are flagged red and those who are not flagged. And we capitalize on the fantastic administrative data on a whole range of great things, student-teacher presence, learning facilities, bureaucrat careers, staffing, finance. The data was fantastic. Was using it for accountability like this the right way to go? This is what the Chief Minister saw. They saw a dashboard of colors of green and amber, and then those dreaded reds. And the bureaucrats did not want to be classified as the cause of a red. Now, what we can do because the data is so rich, is we can go down to the school level, and we can look at schools that were in those greens, but just in those greens. And those reds, but a school that was just in the red and caused by other schools in the district. These two schools that we're going to assess are exactly the same in terms of their history, in terms of the shock that they received. And so we can do a relatively clean comparison of what happens to the school outcomes when the bureaucrat is shouted at, is has reputational damage, has all that de facto punishment. And I'm going to do so by telling, walking you through what happens to the average school. So what you see here in the, let's firstly take the two solid lines. Is schools either side of that threshold, and the red ones are those that just got into red and the blue that stayed on the side of green. What you see is both of them have a shock. This is teacher presence. The teacher got sick, the teacher couldn't come to school in that day. And so teacher presence falls. And so now we want to know in the next period where the red is punished. Do we see a different trajectory? Do we see the more accounted school, do we see it change the way it operates? As you can see, absolutely not. Schools that are punished revert to the original pathway in exactly the same way. This driving through performance management at the center does nothing, it achieves nothing. And then I can also show you the same thing for all schools, which are the dotted lines. The school reverts to where it would have been otherwise. We don't find impacts on any margin of centralized accountability, teacher presence, student attendance, physical facilities, any of the scores, nor do we find on the activities of bureaucrats. So Is this a rogue finding from Pakistan? No. The work that I just showed you was embedded in a six-country case study where we looked at the same things, researchers looked at the same things across 6 countries. And Martin Williams and Claire Lever were here a couple of weeks ago to showcase the endline results. And what they argue is across all countries, data is not so useful as an indicator for allocating top-down rewards and sanctions. Rather, there are huge potentials for organizational learning. But this is not always prioritized. In fact, very rarely prioritized. It's very, very rarely integrated with mainstream civil service routines. So what my co-authors and I did to show what could have been done with that data. Is we follow Alessandro Fennizia's work identifying the talents of individual public officials, and we take head teachers and we give each of them a talent score in their ability to bring teachers to school, in their ability to keep student attendance up. And what we find is that can move the needle. If you get a talented head teacher to your school, that's what changes teacher presence. Let me focus in on the graph below for a moment. What this is is a scatter plot for the talent indices that we create for every head teacher. So what you see on the teacher presence column. Is all those teachers' scores, but then cross-correlated with how they did in other one. So the purple shows us how well does a teacher able to bring teachers to school, but also to raise functional facilities of a school. And what's fascinating is that that is a basically random scatter graph. Whether a teacher is good at getting teachers to school or good at functional facility maintenance. It's not that related. Now, it's quite related, you can see with scores, maybe as we'd imagine, you are good at getting math scores, maybe you can get English scores up. But what's fascinating here is that we can use data to identify the individual differential talents of public sector workers. The comparative advantage of that coarse but large scale data in the public sector is to identify underlying parameters that no individual bureaucrat could do on their own. No public official knows where they are in the large end distribution. Most of those public officials in the top tiers of being able to bring kids to school, being able to get teachers to school, didn't know that they were one of the best in Punjab. So if we give them that data back, well then we can complement management, their management, and the management of people higher up the chain with very targeted data. So a call will come, surely you're not saying we should have no accountability in the public service. That's right. What I just talked about was centralized strongman accountability. But how about being accountable to each other in our departments, in our organizations? The classical way that the bank has done that is through implementing appraisals like the OPE. It's where you set targets. This is what I'm going to achieve this year. And so with some co-authors, I spent the last 6 or 7 years studying what happens when you implement an appraisal system in Liberia. Liberia is a country that had a ravaged public service after the civil war. It was a place where people did not come to work, where people were unhappy, where people turned over. It was very difficult to manage. And so our interest here is whether we can encourage the uptake of the appraisal system as the foundation of public service structure and see what happens. Well, we implement two treatment arms in collaboration with Liberia's civil service agency. One where we try to roll out communications, training and support to get the appraisal up and running, focusing on the role of senior managers and peers. But today I want to put them together and just say what happens when you encourage an appraisal system you put in in this fragile environment. We collected data on the impact of rollout over the next 2 years on attendance, management, quality, experience of the state. And what we find, and here's a regression table, and I want to show you that because People start turning up more. These numbers are attendance figures. Because we gave some accountability, because there is some structure, people start arriving more and voluntary turnover goes down. People stop leaving their units as much. They're also more satisfied. Here's some satisfaction measures. Satisfaction goes up 8% points. And when we ask, you know, why is it you don't want to leave the unit, it's to do with their intention of moving up through the service. And so in Liberia, an environment in which one would imagine it's very difficult to get control, build structure, we find detectable effects, big detectable effects. When we did qualitative interviews. The individual said the system is being introduced in our scope of work is actually good. It helps motivate us. It encourages us a lot. You know, there are a lot of things behind it. But we don't want to just create structure so that people turn up to work. We want people to innovate. We want people to strengthen public officialdom. And so in Ghana, with co-authors, we study exactly this idea, a culture of innovation. How do you imbibe a culture of innovation so that public servants can build the service themselves. This is based off a bureaucracy lab that I told you was starting in my last public service policy research talk, and What's wonderful is it's still going and they're doing stuff. In fact, we have limited to do with them now, but they're still working. And this is the initiation of that, that, uh, bureaucracy lab. But specifically what we wanted to do with them is say, You know, can we change the way that you train public officials to make them more public service interested, excited? Can you make them better initiators of new ideas and generate a better public service? So we collaborated with the Office of the Head of Civil Service, and all we did was we took this 2 week standard training. Now, I'm assuming that most people in this room have done some World Bank public service training. It is a core way that we build capacity in the public service across the world, but more importantly, it's a very large part of the bank's capacity building efforts. So we just added a module, a single module, where we worked with people to diagnose and solve problems based on what they'd learned. To innovate from the bottom up, to conceptualize change, we role played, we gave them a platform to take what they'd learnt back into the service. And here we again implement two treatment arms, one where we take individuals into a classroom and make them into individual innovators in their unit or department. And another where we go to the unit or department and we try to get everyone to innovate. And I want to split the results I show you because the results are so interesting along this line. So we collect data on the impacts of these trainings over the next 2 years on culture. Can we change culture, administrative process, task completion rates, and thank you KCP for making that last two things possible. So what do we find? Again, I'm showing you regression tables. Because I want to show you how strong the effects are here. So if you have this innovator training, what's fascinating is that having a single innovator. Improves overall division culture. What you have here is claims about division culture from a civil servant survey, and individuals say that the teamwork climate goes up, performance climate, fostering new ideas. The the entire unit's quality of culture is improved. And when we do it with trying to train everyone together, when we go out to the entirety of the department, we find no effects. Now, what about idea sharing and generating, generating improved processes? Well, here we, we see is that those who are in the classroom training where we created innovators, well, they feel that there's now more freedom to express those ideas and the frequency of meetings on process increases. And yet, the best evidence we have from the classroom training where we train everyone together. Is null to maybe even negative. And finally, does this have an effect on process itself? Yes. And not just that, if you look at the column at the end, do you remember those task completion rates I showed you at the very beginning? They increase by 11% points. It's noisy, but something's there. We don't find any similar effects when you try and train everyone together. Why? You know, we have some quality evidence. With your superior in the training, it was not beneficial, said one for feedback of individual from from team training. You have to be cautious about the superior subordinate relationships. I did not see the benefit and I could not voice my feelings. You know, each of these impact evaluations surprised me. And they underline it. How much and how little we know about how the public service functions. We've probably been training entire units. For 40 years. Have we ever created innovators? Probably sometimes. Did we do it in the right way? You know, administration can be changed, it seems. And for the better. And even small but well-designed interventions can have impact. And again, that's expost obvious because we're probably so far from the production frontier that small tweaks can push us quite a long way towards it. Yet what works and doesn't is so poorly understood. And for an organization whose primary intention is to influence policy, to me, This is an urgent need for better information. You know, I'm not asking us to respond to any of the IEs that I've done. I would not want us to respond too intensely to any single impact evaluation or set of findings, but surely we'd want to update our priors and invest in this kind of knowledge. Bureaucracy does seem responsive to interventions that build up professional features of the service like clarity and duties, identification of talents, responsibilities to improve procedure, and yet in the economics profession, there is almost no impact evaluation on these things. So is another world possible? I think it's vital because we only have 6 years to achieve the SDGs. For those of us who think about this sort of thing, we've basically run out of time to introduce new programs and policies with uncertain impacts. What we do have time is to go and find the talented managers and the talented public servants and put them where they're needed most strategically. We need a technology that has immediate and sustainable effects. You can move a manager in a month. Something that can feasibly impact those half a billion policy decisions where a dollar has catalytic impacts. And of course, beyond the SDGs, there are going to be global challenges that require an interconnected and long-term approach to policy influence. The battle will not stop here. And what program can achieve that? My opinion is that it is one focused on analyzing and strengthening government. My first policy research talk sketched out how the my public service is a unique and complex place. You should go and watch it, it's good. It showed us how microdata is so important because in such a diverse place, making a single statistic for the entire of a government. Doesn't tell us too much. But what I'm saying now is we have to act on that information. We need to use that information to improve government. And we're not going to influence public policy unless we do so. You know, there are likely to be substantial passive inefficiencies, the resolution of which have substantial welfare gains. So why are we not changing the narrative of the World Bank towards supporting government and its officials to allow them to work more effectively on their decision points? What do we do now, James? Well, Michael gets a, a starring role. He has told us for years. What are you doing for the median official? His pioneering work trying to push us to think about the implementation side of our work is exactly where we need to go, and we can have different flavors. The work that I do is different to the work that Michael does, but I think we both agree that the question we should be answering is, are you supporting a public official to be their best professional selves? That's a question we should all ask ourselves every day. My answer is government analytics. It's to repurpose administrative and survey data from within government to improve public sector functioning. Release the service. To improve itself. And this offers us. A distinct microdata-based way to improve state capacity. It enables governments and development partners to detect areas of improvement. I've shown you how just descriptives show you where all of the least effective agencies are. I've also shown you that tweaks to that administrative data allow us to target where talent needs to be supported. We can work government-wide, but more likely we can work agency by agency. We can draw on a range of sources, so we're not just taking one statistic and using that to drive our effects. But. It means I have to face a very hard reality. It turns out I am not James Bond, and not just because of the good looks. It's because of the immensity of public policy. There are just Too many decision points, too much action for me to be able to save the world. I'm afraid that's true more generally, we are not James Bond. We're not going to save the world by shooting down decision point after decision point. There are just too many of them out there. What we need is an army of capable. Committed and not totally constrained. Public servants, our James Bonds all over the world. So where does that leave us? Well, we're in this building or in a country office. So we're back at headquarters. Once we're humble enough to realize who the real heroes are, we can at least put our lab coat on. And be some sort of cue. One slightly nutty analyst whose intention is to build tools to support field operatives and in our case, public servants. That's what we can do. And if that is our intention, then I think it changes what the World Bank should be doing in policy and research. Let me tell you a little bit about how I think we should play the role of Q. We should ask public officials more, the median public official. What it is that they're facing today and how can we help with the vast amounts of data that we have. We can be a conduit to the information that they need, not that we think they need. We can use bank projects and financing as conduits for building the capabilities of public administration, but based on rigorous evidence of how to do it. We can provide officials with inspiration. Tools and comparative data to help them make better choices, and we can work hand in hand to make them better analysts of public policy. I think that's what DM is trying to do. We are trying to do that in our iterative way. You know, we have this thing called trial and adopt. It works directly with public officials on specific programs, specific problems and evaluations, and therefore we have an incentive to listen, to support capacity building, to help build data systems inside government. Otherwise, nothing will be adopted when we suggest our mid-course corrections. Ariana says it, that we focus on skills development and behavioral change to support the deployment of development ideas as an opportunity to build local capacities for linking data and analysis. My flavor of DM has been the bureaucracy lab. Zahid Hassanein and I have been building it for years and we're still going. We're building a collaborative space for deck, research and operations, the governance practice. But I think we can do more, we can present practical best practice. Myself and Christian Schuster have. Been working on this with Galli and others to create a government analytics handbook. We can share comparative data. We're going to launch, thank you Aisha for all the hard work, the Global survey of public servants, the first comparative data on what the public service looks like. We can place data in the hands of public managers about their units. We can benchmark institutions inside government and show how do you do compare with your peers, and sometimes because this is demanded, we can even show them national benchmarks for governments all around the world. And we should make everything we do. Completely free and completely reproducible, which is what I intend to do with every measurement piece that I ever do. And people like it. My favorite, of course, is back with Nana in Ghana, who said that the survey we did with him was the first time he'd truly seen his service. And why are we not running more experiments inside government? Experiments are good ways to learn. Let's use them to understand how government works. Does any of this work? It sounds good. So what Ravi Samani and I did in Ethiopia was to randomize information briefings like I'm suggesting. We gave it to some public servants and not others. Ethiopia's a a a very important counterpart to the bank. But when you actually ask public officials about the people that they're working on, they get it very badly wrong. 50% of officials that we ask make errors about their underlying objective measures like the population they're serving, the education levels, the number of education. They get errors 50% or more. And so what we did was we provided a random subset of government officials with An information briefing And we then collect data on the accuracy of their beliefs afterwards. Now we do this across the whole of government. Now what would be the classic response? Well, of course, it's local governments. They're the people who know their environment. Actually, the evidence briefings improve the accuracy of their beliefs by an amount similar to decentralization. Decentralization by 1/3 of a standard deviation, the information briefs by 25%. My colleague Aidan Coville and his co-authors have started to work on this, but if we're trying. To update public officials' beliefs, why is there so little evidence in the bank on how to do this? I want to shout out Juan Santini. Who with his co-authors did a great piece of work where he informed mayors about research and is able to increase the adoption of those policies by 10% points. What I'm suggesting, what we've been doing, it does work. But only when it's fully embedded in an understanding of the administration it's trying to affect. If we're trying to influence policy, those half a billion decision points. It's going to be hugely mediated by exactly these things. I'm quoting myself Our job is to not to constantly seek out ways to punish public servants when they get it wrong, and to be honest, we have limited ways of doing so. Rather, our best chance is to invest in helping them get their administration right to give them the freedom to succeed. To ladies and gentlemen, You have a license to respond. If anything of what I've said is resonating, is interesting. Well then, firstly, think of public servants that you work with as James Bond. Give them a sense of autonomy, a high mission orientation, help them search out information on a case by case basis and support that culture of strong professional relationships. And I'm now going to try and give each of you some thoughts on what we might do. And I'm going to separate them into the 3 audiences I think these talks try to talk to. Researchers. Those of us who try to be on the fence or the bridge between policy and research, and then operations. Hopefully, members of DEC can remember DEC. Describe, engage, and cross-randomize. When you're doing work in any field, try to measure the administration that it involves, that it engages with, and add that to global knowledge, don't let it just fall away in your appendices. Engage with your counterparts, not only on the project that you're doing, but ask them how can my data help you more generally. And cross randomize. If you're doing work, then try to bring in an element of the impacts of bureaucracy, because given the data I've shown you, it's likely to mediate it when it gets into public policy anyway. To my lab scientists, LAB link, anchor and best practice. You know, public service is not a place where information flows freely. It's not a market, an atomistic market, and therefore improve information flows between public sector actors who should meet in the Ethiopia, Estonia, and other case studies that Aidan and I have done, tacit knowledge, who you talk to matters hugely and therefore, who are you going to link up today? Anchor, generate or support a community of practice that shapes public service norms and best practice. Support public officials' ability to access that best practice in a way that easily translates to their service environment. Operations, although. I feel shy talking about something where there's so much expertise in the room. But OPS observe, practice and strengthen. Make measures of the administration part of your project as they might mediate your success. Put one performance indicator based on public service. Practice, trial different approaches. Trial and adopt. And share your results with us and strengthen. Always think about how your operation is strengthening the administration you're working with, so that the next operation might not be as mediated as yours will be. People must be very upset that I've said they might not be able to be James Bond. But you can be actually. There is of course one place that World Bank staff can be James Bond. And that's at the World Bank. You know, there's no administration actually that has been more resistant to my attempts. To undertake analytics on it than that of the World Bank. I've been rejected at every turn. But this is in fact the one place where we could be James Bond. This is the administration we're most equipped to truly change. But we have to focus, focus on changing the administration of the bank. Working on individual decision points will not be enough. What little data I have on. This place that I've been able to gather shows that there's a set of unique challenges here. The bank's employee survey has a few questions that overlap with the questions the rest of the world asks and that my collaborators and I have harmonized in the global survey of public servants. Here's a few of the overlapping questions and indicators of where we sit in the global distribution. Although because so few countries ask questions like ours, there's only a few comparators. We are, as many of you would agree, a motivated bunch. We sit at the top of the distribution for a number of questions related to how motivated we are by our work, both deck and the group as a whole. However, When you look at how we organize ourselves, things look worse. We are at the bottom of the distribution in terms of how we collaborate. And how effectively we build pathways to capitalize on the bank's talent. We should change that. Sometimes we hold at arm's length the idea that the World Bank is a public administration. It is. And thus everything I've argued here today is applicable to the bank's own administration. Well, it's time to wrap up. I hope something that I've said here today resonates with you. If it does, please think about it. I believe that fixing bureaucracy is something. We should all be part of. James Bond will return, he always returns. How are you going to help him save the world? Thank you. Super. Thanks, Dan. Yeah, we, we didn't, there are no martinis outside, just set your expectations. Thanks, Dan. Uh, over to you, Arturo. Um, so I'm not so sure where to start. Uh, uh, really interesting. Let me actually start on the, on, on the last part because I, I think we are. We are looking to some of the issues from, from a similar perspective. Uh, we, uh, in governance, we, we work with institutions across the world, the world, and we try to fix things, uh, uh, how to set up their systems, how to make, uh, they, their human, manage their human resource, how to manage their budget, etc. And, and as such, we, we have experts who, who work on the, on the civil service side like your colleague Saheed. Uh, and also people who work on, on, on gov tech issues in how to manage the information produced by the government and uh I, I would say 3 or 4 months ago I had the same concern that like that you were having, and I, and, and, and actually it was not an original idea of myself. Let me tell you how I come to. To a to a similar point of should we be looking at how to help this institution, uh, to work better, right? And, and, and I remember I, I was remember a failed attempt that was intended 14 years ago, um, you know, during, during the, uh, the financial crisis in 2008, there were several banks who were the first who were hit. The first one was Bear Stearns. And uh, like many, like many banks they have people who are experts on two different kinds of financing. The ones who, who deal with the clients were helping clients to raise money into a different spectrum of, of, of, of challenges, and there were people who were raising money for barristers, their own treasury. So when, when the crisis hit and it was incredibly difficult for, for, for investors to raise money, somebody say why don't we ask our client face team, our investment bankers who usually go and raise money for a company which is in uh passing troubles to raise money for us. So they asked their, their, their in-house experts to, to look for a, for an internal pro problem, and that's what we're proposing here so. Interestingly enough, I asked my team 3 or 4 months ago to do the same, so they work, and I asked them to prepare a 2-pager about how to fix the batch rotation and the bank and how to get the information that was there. So, so were you, I invite, I invite a Sahid which uh was part of them, and we prepared the two pager. I think, I think we can, we can help to solve. We came from a similar angle. We probably, we, we're probably young because I sent that the two pager to somebody and I'm not sure if they have an open it, uh, but, but, but, but that's that, but I mean, as I said, we are, we are addressing some of the issues from a similar concern. Having said so, I have to confess that when I said, uh, I saw that the cut theory of development, I, I thought it was about Tobin's queue. And, but this, this is, uh, so I, I wasn't sure how to, how to, to grasp it. Let, let me, let me now, uh, move to some aspects which I think um. Uh Required to probably to to to probably uh contextualize it. uh I, I want to start with one of the of your statements through through the presentation you said uh we urgent need, we have an urgent need for better information. And that I couldn't I I couldn't agree more. Now the question is what kind of information? And um it could be very complex information if the problem is really difficult to crack, but it could also be very basic information. And in, in some aspects, what we are missing is very basic information. Just, just trying to do this descriptive statistics it's gonna be, is, is gonna be, is, is gonna be uh uh really important. And then. A different question is what we need that information for. And what I found very commonly both. In the ply work and here. Is that people tend to jump to try to do something fancy without first understanding what is the problem that we are gonna try to solve. And, and we, we need to have a characterization. Of of that of that problem and sometimes going through the process of of generating the information it's actually it's actually interesting and what we may find from one context to another is different so so let me give you an an an actual example um two different phases in my career probably during my whole career I have been concerned about the issue of how to collect taxes right? So in my la right now is it is an important part of the governance mandate. Before that was how to collect federal taxes in, in, in Mexico as finance minister then. In my first incarnation here at the bank but before that also, also when I was finance secretary of Mexico City many years ago and uh and and we have and we and we face. Some serious problems at the time that was almost 20 years ago how about how to collect three different sets of taxes property taxes, water charges, and car property uh uh uh and car property taxes and through the time I I have been trying to replicate some, some of, of these studies, um. Uh, but let me start with one example which is what, what made me think that we have to be careful if it's just about providing, uh, public servants, how was your work to, to be whatever they could, all the, all the best they could, they could be, so, so. At the time in Mexico City, the same taxpayer have a different response to which taxes they were willing to pay for whatever reason they would pay the property tax at a higher rate than the water charges, and the last one was the car property taxes, right? And that became really really uh uh a problem for me and I start thinking trying trying to get some basic data actually. Uh, which was not, not easily available. Why is people not paying taxes? And I, and I was getting desperate, uh, and by the way, that's, I'm gonna bring me for a quick comment on, on, on your issue of centralized accountability. And, and I think there's that part of the problem that you may find with a centralized accountability is that they are asked, uh, when you have someone which is very powerful and asked to solve a problem that's not sufficient if they don't describe properly the problem. So, so I may have been partially in that in that in that seat and I was, we need to collect this this tax we need to collect the tax, but I couldn't understand why it was a problem why people were not collecting the tax and I was in a meeting with people who who all all all of them were were responsible for that, right? One were uh responsible or actually the database another were to to do the uh uh the the the the legal claims, etc. And then it occurred to me something which was not very systematic, but I thought it would help me to to understand the problem. Uh, all of them, the public officials sitting on that meeting had their cars in the parking lot, right? So I asked someone, why don't you go down and check if they are paying the tax. Right, the ones who were collecting the taxes, so somebody won, it was not a random experiment, right? Like it was so they come and say, well, actually they are not paying their taxes, right? So, and so part of the problem is that people who were in charge, the senior people who were in charge of collecting the taxes, didn't themselves believe. That they have to pay it or that they or the or or or so so so it's not always about how to to to make them the best that they could uh that they could be you need to you need to understand if they are actually uh uh. Let me give you another example now with the property with the property taxes um because the process of generating the information opens different questions. So, um, so when I tried to revisit that those kinds of issues when I was back at the bank around 7-8 years ago, um, uh, I'll, I'll try to figure out now why people were not paying the property tax, and, uh, at that time there was already already literature, particularly from the UK. That say that if people knew that their neighbors were paying the taxes there was a a higher propensity for them to pay the taxes so so that the the underlying hypothesis is that um uh there was some sort of uh of social uh commitment. And, and that that by appealing to that people will fulfill so rather than sending them a letter saying you, you need to pay your taxes so we're gonna go and collect it, you will send a letter that say 80% of your neighbors already pay the taxes. Why don't you do it, right? And so, so, so, so I was replicating an an actual experiment that I did many years ago. And uh when we were sending letters and we we we did it this way right? We divided um the letters uh by one letter was written in a very harsh tone right? We have detected that you failed to pay your taxes we are gonna give you 15 days to pay it. And the tone and in Spanish, you know that uh you could address somebody in a formal way that's sed or, or, or in a friendly way too. So that was in the user in the very formal way, right? Then we we prepared another set of letters, which was. Addressing them in the two-way and say, you know, but the property tax to property tax we collect x amount and that serves to pay the subway system, the firefighters, etc. right? Trying to appeal to what what cut services. Could the public perceive as good, not the police, because sometimes if the police comes and smack someone. So there was another, another subway line who have a problem that we didn't put that subway line, right? Just think that people that people thought it would be and then we have a control. The control letter was a letter that they will send, uh, uh, in a standard way. And I'm following a very formalistic uh uh a formalistic approach that letter uh was starting with a legal basis right according to the code that we are in charge of collecting these taxes and then it will say all the places that it could could be paid so. You could pay you could be paid in all these banks it could be paid in Walmart. It could be better so it has a lot of information so to fit that in that one letter they had to use a very small font, right? So it was impossible to read that letter, right? So we send them right? and and then one of the letters also we may have a different division one was sent by the by the fiscal attorney which in principle has a a higher a higher authority. So the result was the opposite of the UK. So the harshest the letter. The more collection, right? Actually, the letter that is impossible to read, collect more than the nice letter, right? And so, so that gives you an idea about what should be. The optimal Tax collection uh approach which is be as hard as you could be, right? But for me also raised a different question which I haven't been able to tackle right? which is why is this happening? And, and, and in my mind there were two options. Uh, one is, uh, particularly 20-30 years ago, uh, a lot of the revenues from Mexico were coming from were all oil related, so that this idea that you pay taxes and get some services was broken. We were getting services without paying taxes and that's why probably the people who were actually in my team many years ago didn't feel that they have to pay taxes somehow the money was coming back, right? But that's a different problem than than understanding why people were paying taxes or no it's understanding what what was their role in the social contract. And that was one hypothesis the other hypothesis is. Well maybe the society have a very. A very stre stressed relationship with authorities. They are not willing to cooperate in anything, only if they are forced, and that's probably. And even much more serious problem right? so that's what I thought uh. If I had time at the time those are the things that I would have wished to investigate so but uh but I wasn't trying to find that those problems though I stumbled. Into that into that by just trying to figure out uh how what were the the general dynamic so, so the point I'm trying to say is some of the questions and some of the relevant questions and even some of the research questions are, are popping up just by uh by trying to gather uh uh uh to gather information. Uh, let me just make 11 more, uh, one more comment, um. Uh, which is, um, So I really like your idea of the innovation. I have all, and I think it makes, it makes, it makes a difference. I'm only, I have always been worried. To how, how, if it's possible within a government and a bureaucracy to move to a permanent innovation mode actually bureaucracies are about the opposite bureaucracies are about finding a way to do something and just keep doing it. That's the caricature of bureaucracy. They, they, they refuse to do it in a different way. Right. And as some someone told me once, this is bad and good. Also, when bad governments come with what they keep doing what they are supposed to, uh, they are supposed to do it. So, so this innovation could be good and disruptive when you change that process, but then it has to be codified in a. Uh, in, in a, in a process, and bureaucracies are good about it. When, when I first was a general director in Mexico, they sent a team to try to understand what we were doing in each process, and they, and they, they start going desk by desk, right? And at some point they call me and say, we found something which is. Which is a little bit of concern. These guys reviewing some papers, check them, classifying two different parts, put a seal to one and say, And they don't use that thing for anything for the last 15 years, right? So some somebody forgot to tell that person that that thing that he has been doing. So, and, and in, in, and actually they say, please don't let him know. Yeah, it's, it's gonna, it's gonna be, it's gonna be traumatic, right? So, but on the one sense that's an example of if you codify it, people just keep doing it. Right, but you have to ask why they are doing it, so let me stop here. Thanks Actor, and, and stretching the James Bond analogy, um, and to go on that point, which is the number of movies where he goes rogue is is a fair share of them. So working in a bureaucracy in order to get things done, do you have to go rogue? Anyway, um, I have a few questions in the chat, but let's me just take a sense of the room if people have questions or comments. Um, OK, let me take these two, starting here and then there. OK, so from a, I'm a very operational person. I'm the implementation guy. So from your talk, I gathered that may be 3 areas of implementation that we can incorporate in our project designs. One is the implementation of some kind of performance management systems. The second is some kind of analytics, implementation of analytics, data analytics. And the third is some additional training to the civil servants, training on diagnostics, training on innovation. So this is very positive and it looks like it has impact, but on the other hand, we have a large number of failed civil service reform projects in the bank. So how can you integrate your findings with the lessons learned from our past failures? Thanks. Thanks. I have a question about training innovators or creating innovators. Uh, do you have any insight into what, what does it take to make an innovator? What's the recipe? Like, is it, you know, is it, do you provide training on administrative skills, some part, intrinsic motivation, etc. Um, and secondly, along those lines from your data, Is there a cutoff point for competency among civil servants and bureaucracies? I'm thinking like, I'm working in Papua New Guinea in a very capacity, capacity constrained environment. So, when I work particularly down at the level of a school or head teacher or a district, right? They often lack The basic competencies to fulfill their roles, whether it's the, the, the training or what, uh, or the diploma. Um, so, like, at what point do you, does a public servant just need to have the capacity, uh, and, and the competency to do their job? And at what point do you train them to be an innovator? Like, where's the cutoff? So let's turn it back to you, Dan, if you want to sort of react to Arturo and then see what you can react to those questions. Yeah, very quickly, I mean, Arturo. Thank you very much for being here for discussing and for your ongoing support. I think, as you say, this is something the practice is doing at a a more vibrant scale than ever before, and I think I take my hat off to everyone involved in that. You know, in terms of the idea that we're freeing public servants is not what I'm getting at, is that we're giving structure. You saw structure is what people want, but I think we have very little sense of what that structure should look like. We've been taking pro forma ideas for a long time without really experimenting with them or, you know, utilizing, uh, even measurement of their long-term effects. And so here, an appraisal reform that was sold to the service was more effective, you know, than one, where you're just basically leaving it to centralized actors to impose it. That's just one example. So structure's important, and I think that's also true of your, your question around, you know, should we take the conservatism of government and convert it purely to an innovation. I think that's the fantastic thing about a public administration because it locks together actors with very different incentives in a way that many institutions don't. And so there's nothing in, you know, an innovation, uh, focus that means that everyone has to be an innovator. There are some aspects of government that should rightly be conservative, and yet, why not have someone arguing with them around the table? And over time, it's having the. Institutions that effectively mediate that conversation in government that's going to work. It's a little bit like DC and the bank. DEC is supposed to be an innovator, creating new things. And so we have a machinery that allows some of DEC's work to influence and others that are more like a laboratory. And so I think it's the machinery that sits above those that's going to mediate its success. You know, um, what you were saying about surprises, how many surprises you've had in public policy is exactly what I found. This is a very different place to the worlds that we have been able to gather information on before. And so I would say that pushes us to be more innovative because at the moment we're probably doing things that aren't that effective, and we're doing them systematically. I would say we need to systematize the innovation. It goes a little bit to your idea that civil service reforms have failed. I don't know where that idea came from. Do you, you know, I don't think we have very good evaluations of civil service reforms, you know, there's obviously classic literature on the IEG report that sort of said these things don't work. If you talk to the IEG editors, they actually said that's not what we were saying. And we've had that, and I have met with them, and what they said was we need better diagnostics to identify what works where. So I pushed back on the idea that civil service reform doesn't work. Maybe writ large, trying to get the entire service to completely transform. Why would we need to do that? We can move agency by agency. The. Many small changes that come from the Federal Viewpoint Survey in the United States make for a very large change in the way that the public service works in the US, but it's individually based. It's, it's one manager changing one thing at a time. But it's a technology that's been made available to all managers that's making the big change. So, you know, I, I would say we need to, to push back on the idea. I don't think we know that civil service reform. In its flavors that were say in the IG report didn't work. We just don't have the data. And then going back to the sort of the idea of training and, you know, to what extent do these things that have surprised me, this is very fresh. I mean, I'm now doing 3 other IEs all around in training because I'm like, We don't know, we better go and start finding stuff out. One of those IEs has films of Ethiopian public servants being James Bond characters, not quite, but you know, being heroes and and get making souls. Is it a narrative that we need to change? Other IEs are much more structural. And so, you know, I can give you guesses at the answers to those questions. One is, you know, capability is, I think, something that is kind of imposed by us because my kids are very capable of changing things that they want to change. As long as you're asking the public servants what they think are the problems and helping them, and that Ghana, we didn't say this is the process you need to change. What they said was a lot of de jure processes have been put into Ghana's public service. We just don't do them. And so I think 70% of all of the change innovators was to move the de facto to the de jure. And I would say that's going to be true across most of our client countries. There's very nice de jure rules, what you're supposed to do, and yet most people do not follow them. And so what the public servants wanted to do was basically shift their departments towards the de jure. So I think if you talk to your teachers, you talk to your education officers and the local governments and say, what do you think's the big bottleneck, let me give you some tools that any level of capability can use to solve that problem, I think that's the way we think about this, not your school should be teaching with this curriculum, let me help you get there. So I can make more comments, but I'd love to hear if there's a couple more questions here. So let me call on Juan. Juan, are you still online? Juan Marron? You had a question in the chat. Do you wanna come on screen and ask your question and make a comment? Can you hear me? Go ahead. Yes, we can hear you. Yeah, no, thank you, Dan. It, it was a thing. I, I think that a little bit of a call to the, to the governance, um, global practice and education, global practice in education you have heard Jaime talking about a lot about implementation capacity. Uh, and it has been a struggle, right? So, so we did like an analysis of how many different, um, education, uh, multilaterals, bilaterals do the training and it's a complete failure of what we have been taking as, as, as a success. And, and a lot of, of that is what you were saying is that we bring like public officials training on, on something but honestly I've been in Very different types of countries and that is just like a way to get per diems, right? So, so it, it is, that is the reality. Um, so I think one place where, where we could work together, um, uh, from the research side, from the operation side is, is, is how to measure capacity but in a way that is a little bit faster. We don't have the time, we don't have the, the expertise and, and, and, and the needs are there to just get some analytics on, on, on capacity. Capacity for what, for example, is the first thing that that That, that I asked. So that's one. The second one is I really like Arturo's point on what the problem is, what is the problem, right? So that we're trying to solve, but I, I see many countries getting in so, so much analytics, but then they don't know what to do with the data. So why are we collecting this data for, right? So I think, I think your call for analytics, uh, uh, matches and aligns very well with what Arturo was saying about, um, um, uh, innovation. Um, and then the other thing that struck me a lot that, that I think that I feel that as I advance in my career at the bank that I feel more frustrated with and, and have voiced here is this division among people who are doing impact evaluations and research and operations. I was very saddened by seeing like how your recommendations are different from these, when, when, when actually you showed up very Nice presentation saying it's like we don't work well together, right? So, so that, that's what we show in the analytics and this is a call for all of us. It's like how do we start bringing you uh this impact evaluations as part of our preparation, as part of our, our, um our project preparations. I think that is where we can really have like like impact. I'm not saying we shouldn't do A very good impact evaluation, but we, we need to bring a lot of the, of the insights of all these research into our operations. Otherwise we will continue having great, great analytics, great research that honestly many of us don't use. So, so not to be critical, just a call to work together on, on bringing operations and research a little bit more closer. Thank you. Thanks, Juan. And before we start closing up, um, uh, there's a question by Charlie. Charlie, do you wanna come in? It just says Charlie, so I have no context for that name. It's about to end and uh thanks for a fantastic invitation. Uh, we can't hear you very well. It's cutting in and out, um. across the street with the computer on the screen. Sorry, Charlie, we, we, there's something amiss with your mic, it seems. I'll give you a minute. Otherwise I could read your question out from the chat, but I'd rather you, if, if you have another mic. The. Then What's that? Try to switch my mic. You hear me? Very quietly. OK, Maybe I'll just, I'll just read out the main points from your question if that's OK. Um. So, The question, the emphasis is that there's a way to foster individual innovation leaders can be seen to be counter to other things that characterize better performing bureaucracies, sprit de corps, the peer pressure and support. How do you square those two issues, um. He says we don't really want rogue bonds. Um, second question is that the Pakistan example of individual skills would, would presumably have so many exogenous variables that are more determining than individual management characteristics. Even if one can control for the individual, would that same person really be as effective in another part of the country with another set of variables? And then he has an operational question, which I think I'm actually also curious about, which is much of what you suggest would fit the old Lil type of operations, and for those of you under 40, that's learning and innovation loans. Um, which are now defunct. Our operational management incentives and our legal framework are not well geared to projects with lots of adaptation, which is something I know we all struggle with. Um, OPCS would benefit from this. OK, um, any last questions from the room before I turn it back to, to, I'll give Artu, I'll give you the last word, OK, um, a question here, and then then and then Arturo. The. Just a quick question on how the innovators that were trained were selected because that seems super important and I think I, I missed that for the treatment that worked. Wonderful. Um, I think the idea that we have innovators, rogue, going rogue is worrying to everyone. I think an a successful innovator, the way it was sold here was very much that you're going back to be part of a unit or division that you are going to change. And so therefore, innovation and innovator here, in fact, is binding the individual back to the service that they're trying to change. So in some sense, you could imagine James Bond. Going back to MI6 and reorganizing it would actually not have been going rogue but bringing really good consultancy that would have been useful for the rest of the service versus going off and on a vendetta that he was worried about. And I think we're talking, especially in the Ghana case, but I think more generally, we're talking about that one. The trainees are those that were sent to. The standard public service training, this is a kind of rotated part of every year, so, um, the random, randomization happened as you hit the training institute. And so you're comparing, this is sort of external, these are folks who would have come to training. These are the folks who would have gone through their annual or in our case, 2 years' worth of training. Um, but I think it comes back to this idea of, you know, how do we embed innovation in our operations? And I would say that right upfront, you know, a lot of operations are not taking the evidence that already exists. And so I think just starting every operation, sitting down with a load of deck colleagues and trying to understand what evidence and data we can already apply. Secondly, is then saying, well, what's useful for us to know for the next operation in 4 years and starting the impact evaluations then, and absolutely, Juan, you know, a lot of what we've done, so for example, in Ethiopia, is to do short IEs. So we're doing a report for the Ethiopian Management Institute now. While the longer term evaluation goes on, that's how we did in Tanzania. And so I think in some sense, what you should expect from your colleagues in DC is this portfolio ability that we're using a whole range of tools to build the knowledge that we need, then sitting down with the colleagues before they start the education program in 3 or 4 years and bringing that knowledge back in. So it's using our project today as a platform for doing better projects tomorrow. Thanks Dan. So Arturo, last word to you. So, so, so very quickly, first of all, uh, analytics basic basic data, that's super important, right? And it could come from different ways. It could come from just computing raw data, administrative data, which is already there from getting insights from, from, from, uh, from studies then. Having the right definition of the problem that we want to address that's critical that that that's not that's that's not negotiable and let let me just give you one example many years ago I was manager here. And there was uh this uh this uh grants that the one that the, the, the bank was gonna guard to, to junior staff who come with an innovative uh way to use telephones to solve a problem and then I got one proposal from one staff. And I keep asking her, what's the problem you try to solve? And at the end, it was clear that she just wanted to use a smartphone to do something and she was shifting the problem as I was questioning. So shouldn't be about using phones or an RTC or behavioral economics. first you need to define the problem first and then you figure out which is the which is the right answer. Great, thanks. I think we've thrown out a lot of challenges here, both, um, sort of in our engagements, in our internal, uh, organization, um, and I thank both of presenters discussing and thank all of you for, for coming in the room and thanks for you, uh, online as well for, for engaging in the questions. And with that, let me call it to an end and just join me in thanking the presenters.

showAllTranscripts

no

duration

PT1H24M36S

scene7File

worldbank/RoggerPRT

scene7Domain

scene7FileAvs

worldbank/RoggerPRT-AVS

title

RoggerPRT

description

RoggerPRT

showTimestampAndTranscript

yes