col-xs-12
col-sm-12
col-md-12
col-lg-12
col-xs-12
col-sm-12
col-md-12
col-lg-12
videoType
dynamic-media
videoDmUrl
https://delivery-p136806-e1377785.adobeaemcloud.com/adobe/assets/urn:aaid:aem:8e963298-a5b2-4a04-90b2-10309afc2839/play?assetname=Measuring+Dev.+2022+Lightning+Talks.mp4
keyFrameImage
timestamp

00:02 So we're gonna have 6 lightning talks,

00:03 uh,

00:04 uh,

00:04 to,

00:05 to follow,

00:05 uh,

00:05 session,

00:06 uh,

00:06 session 4.

00:08 I'm gonna MC,

00:09 uh,

00:10 the,

00:10 the lightning talks kinda get things,

00:11 uh,

00:12 moving along.

00:13 Um,

00:13 each lightning talk,

00:15 uh,

00:15 is gonna be 7 minutes,

00:17 and I'm gonna be mean to you if you go over time,

00:21 um,

00:21 but you know,

00:22 I hope that they're informative,

00:24 uh,

00:24 in,

00:25 in,

00:25 in a short,

00:25 uh,

00:26 in a short amount of time.

00:27 Um,

00:27 OK,

00:27 so we're gonna start with,

00:29 uh,

00:29 Marissa Noii from ID.

00:31 is 42 and I'm gonna let kind of the panelists

00:33 up here

00:34 get back to,

00:35 get back to their seats.

00:36 And just to quickly note that the reception this evening at 5 will be in the atrium,

00:40 which is up two floors,

00:42 the main,

00:43 um,

00:43 the main floor of the World Bank ground level.

00:45 So

00:46 that's which is exciting I've been told.

00:49 OK,

00:50 awesome.

00:53 Yeah.

00:55 And

00:56 if you

01:00 Alright.

01:01 Hi everyone.

01:02 My name is Marissa Nowicki,

01:03 and I work with Ideas 42.

01:05 Knowing I'm the only person keeping you from wine right now,

01:08 I'm gonna go through this quickly,

01:11 um.

01:11 Uh,

01:12 so today,

01:12 I'm gonna just give you a brief introduction on who we are,

01:15 who IDEAS 42 is.

01:17 Um,

01:17 we're going to talk briefly about three different case

01:20 studies and projects that we did during COVID,

01:23 um,

01:23 and how we pivoted to use mobile data collection

01:26 very briefly,

01:27 um,

01:28 and then some key insights that we learned from this experience.

01:32 So for those that don't know,

01:33 Ideas 42 is an independent nonprofit firm.

01:38 It was originally founded in 2008 by a

01:40 group of academics that worked in behavioral economics,

01:42 and they wanted to make sure that their work was applied to real world situations

01:47 and used for social good.

01:49 Essentially we study

01:50 human decision making and behavior.

01:55 Um,

01:55 and what we do is first we help define a behavioral problem.

01:59 We study the context

02:01 to understand why the desired behavior is or is not occurring.

02:05 Um,

02:05 we design a behaviorally informed intervention,

02:08 we test that solution and then we scale successful interventions.

02:12 And throughout our methodology,

02:14 data is crucial.

02:15 It's important to identify the behavioral barriers,

02:19 to create effective behaviorally informed interventions,

02:21 and then to make sure that our designs are working.

02:24 Um,

02:25 and during the COVID-19 pandemic,

02:27 we needed to utilize new and creative data collection methods so

02:30 that way we could pivot and continue doing our work.

02:35 So I'm gonna briefly talk about 3 case studies where we pivoted,

02:38 um,

02:39 and in these instances,

02:41 instead of doing our normal in-person activities,

02:44 we decided to do IVR or interactive voice response,

02:47 which is essentially when a pre-recorded message,

02:51 um,

02:52 is spoken to you and then you click a number which we've all interacted with,

02:55 you know,

02:55 uh,

02:56 for English,

02:56 please press 1.

02:58 I think we've all,

02:58 we've all experienced that and then also telephone interviews.

03:03 So the first one is from the DRC,

03:05 um,

03:05 and in one of our projects,

03:07 we are working to help beneficiaries of cash transfer programs,

03:11 um,

03:11 use this money to build resistance against economic shocks.

03:15 We knew we needed a really large sample for

03:17 this to run a fully powered randomized control trial,

03:19 so about 5000 was our goal,

03:21 and that that would require a lot of resources.

03:23 And as this was newer for us,

03:25 we decided to first do a pilot.

03:27 Make sure that we could

03:29 get the reach that we actually wanted,

03:31 so we ran a pilot IVR survey with about 200 randomly sampled people

03:36 to assess baseline response rate and if there was

03:38 anything we could do to improve that experience.

03:40 The good news was about 67% of people answered the phone

03:44 and about 20% completed the survey.

03:46 And a few key insights from this,

03:48 um,

03:49 were

03:50 that,

03:50 you know,

03:50 there were some technical difficulties,

03:52 which is,

03:53 uh,

03:53 I think a common theme that we want to be mindful of who can participate,

03:56 who might have a little bit of difficulty participating.

03:59 Uh,

03:59 some interesting notes on the language,

04:02 so the local language Langala was preferred for speaking and listening,

04:05 whereas French was preferred for written messages.

04:08 So having an option to have it in the local language,

04:11 but then the option to switch to French if they would like.

04:14 We saw that times of day,

04:16 um,

04:16 and the days of the week that we called were really important,

04:19 so calling during weekend work,

04:21 weekday afternoons

04:23 had the best response rate.

04:24 So understanding when people wanted to be called,

04:27 what languages were the most important,

04:29 and then of course offsetting any potential costs,

04:31 we did this through airtime so that people,

04:34 uh,

04:34 did not have to pay anything to participate.

04:36 So we use this to refine our approach and we

04:39 did ultimately run the RCT and got the 5000.

04:42 Um,

04:43 completed responses,

04:44 so that was a success that went well.

04:47 Um,

04:47 around the same time,

04:48 we were also working in Kenya,

04:51 um,

04:51 and prior to the pandemic,

04:52 we had been in the middle of working with the Kenyan

04:55 government to enhance the impact of social protection cash transfers,

04:59 and then when travel restrictions were in place,

05:01 our work had to stop.

05:03 Um,

05:03 at that time we had been in the process

05:06 of rolling out a wide scale randomized control trial.

05:09 We had already done the trial in 3 of the 4 intended counties,

05:13 um,

05:14 and the data collection was scheduled to begin a couple months later

05:18 and so to make the.

05:19 of the situation,

05:20 we decided to do something that was a contact-free approach,

05:22 so using mobile data collection this time through

05:25 phone interviews,

05:27 um,

05:27 and the goal was to assess,

05:29 among other questions,

05:31 we also had to pivot our research approach and our questions,

05:34 um.

05:35 But the goal was to see if the

05:37 behavioral interventions that we designed in a pre-COVID context

05:40 were still useful to those beneficiaries during the pandemic.

05:44 Uh,

05:44 and we were able to see using this,

05:48 uh,

05:48 using this tool that people who had been included in the interventions

05:52 were more likely to have concrete savings goals and were

05:54 more likely to actually be working towards those savings goals.

05:57 So although it wasn't a full replacement for in-person surveys,

06:01 it was a useful tool to help us see,

06:03 uh,

06:04 that there was an effect.

06:08 And then in the last project I'm just gonna briefly talk about,

06:11 we've been working with the World Bank to adapt a behaviorally

06:13 designed business training program which

06:15 we call the Financial heuristics training

06:18 to an audience of female small business owners in Ethiopia.

06:21 And initially we would,

06:23 or what we would usually do would be

06:24 to go to Ethiopia and do contextual reconnaissance,

06:28 um,

06:28 some behavioral diagnostics,

06:30 carry out the interventions in person,

06:32 but of course we couldn't do that,

06:34 so we had to adapt,

06:37 um,

06:37 and part of what we did was we hired someone

06:39 locally to conduct interviews over the phone and see,

06:43 uh,

06:44 get some baseline data,

06:45 see how we could refine our approach and how we could.

06:47 Adopt

06:48 it,

06:49 um,

06:49 and there were lots of lessons learned about this project,

06:52 about how people appreciated callback numbers and options,

06:55 um,

06:56 actually the value of just a step up from IVR but

07:00 giving them a little personal interaction was really valuable to them.

07:03 Um,

07:03 they appreciated it and of course this wasn't again a perfect substitute

07:07 but

07:07 was useful for us to collect useful data and then

07:11 pivot,

07:11 um,

07:12 and adjust our approaches.

07:14 So just I realized that was a lot of information very fast,

07:17 um,

07:18 in summary.

07:19 This is a really good tool to have in your tool belt.

07:22 Um,

07:23 I don't think this is a replacement for in-person data collection,

07:26 but it is a good way to complement data you're already collecting,

07:30 or

07:31 You know,

07:31 in the unprecedented times we're living in,

07:33 if you need to have a stopgap measure,

07:34 this is a good way to,

07:36 um,

07:37 to achieve what you need to achieve.

07:38 And then just some simple considerations that we found useful to think about

07:41 if you're considering using those tools were

07:44 the sample size,

07:45 the level of detail needed,

07:47 and the potential value add

07:49 of personal interactions.

07:53 All right,

07:54 thank you for everyone.

07:55 Uh,

07:55 if you have any questions,

07:57 feel free to check out our website or send me an email.

08:06 myself.

08:09 OK,

08:09 so our next speaker,

08:11 uh,

08:11 is Vivek,

08:12 um,

08:12 and would you mind pulling up the,

08:14 the

08:16 link,

08:16 link.

08:18 OK,

08:19 and this is what you wanted and needed.

08:22 Yes,

08:22 uh,

08:22 I'll start with the slides first though.

08:24 OK,

08:24 great.

08:25 Just on the other.

08:26 Thank you.

08:28 So when you wanna switch it,

08:30 just like slide it.

08:31 Perfect.

08:31 All right,

08:32 Vivek Saranni from Atlas AI.

08:34 Thank you.

08:35 Hello everyone,

08:36 I'm Vivek Sarani,

08:37 head of applied data science at Atlas AI.

08:39 Uh,

08:40 before jumping into

08:41 some of the technical components of the talk,

08:43 a quick word on our company.

08:45 Since,

08:45 uh,

08:46 I'm not affiliated with the university or the bank,

08:49 we are a software engineering company that uses

08:52 remote sensing,

08:53 uh,

08:53 machine learning,

08:54 and,

08:54 and ground data sources to build

08:56 high resolution data sets and then translate that into

08:59 urban and rural applications for both the public

09:02 sector as well as the private sector.

09:05 So in this talk,

09:06 I'll focus on this idea of where people spending their time and money.

09:09 It's shown up in many ways

09:11 today,

09:12 but the main difference I would say between many of the talks today and this one

09:16 is that we're particularly focused on urban service design.

09:20 So how can the public sector or private companies focus on Where to

09:24 increase capacity or deliver

09:26 additional services.

09:27 So probably Alice's talk comes closest

09:30 to this and thinking about transportation applications

09:33 in the Lagos context.

09:34 And so the,

09:35 the,

09:35 the easy key question to hold on to in,

09:38 in this,

09:38 this idea is where are people spending their time and money.

09:42 And we're looking at mobility data in the Nairobi

09:45 context.

09:46 We're also combining this with high resolution data sets,

09:49 uh,

09:49 for example,

09:50 the asset wealth

09:51 index,

09:52 which is an indicator of poverty.

09:54 In,

09:55 in Nairobi,

09:55 this is a proprietary data set that we built ourselves,

09:58 and then curated sources on urban,

10:00 urban built environment,

10:02 for example,

10:03 Google,

10:03 Google open buildings,

10:05 but also other types of scraped information like points of interest data sets.

10:10 And then a number of the methods which uh researchers in this room are,

10:14 are familiar with,

10:15 uh,

10:15 including

10:16 safeguards for,

10:17 for the data

10:18 and then also some of the optimization techniques for,

10:21 for modeling.

10:24 So,

10:24 uh,

10:24 I'll skip over some of the contextual slides since,

10:26 uh,

10:27 at this point in the day we're all very

10:28 familiar with some of the challenges and issues,

10:31 but again,

10:32 uh,

10:32 the idea that I'll focus on here in this talk is

10:35 that we're interested in not just the location properties of mobility.

10:39 Also the interaction of that mobility information with the built environment,

10:43 whether it's physical assets,

10:45 assets like roads and transportation infrastructure

10:48 or built places,

10:50 stores,

10:50 retail environments,

10:52 homes,

10:53 or other types of facilities.

10:58 So,

10:58 in,

10:59 in the study,

10:59 we're looking at 3 years' worth of mobility data for all of Kenya.

11:04 Uh,

11:05 but for today,

11:05 I'm focusing on

11:07 about 3.5 million observations from 1 week alone.

11:10 And within the Nairobi envelope,

11:12 that's about almost 2 million

11:15 observations from 150,000 unique devices.

11:19 So all the caveats of sample bias and the,

11:22 the,

11:22 the,

11:22 the limits on who owns smartphones

11:24 obviously hold in this case as well.

11:27 Ah,

11:27 but we're gonna try to dig a little bit deeper into them.

11:31 Uh,

11:31 the proprietary Atlas AI data is,

11:34 um,

11:34 1 kilometer by 1 kilometer resolution,

11:37 uh,

11:37 indices on asset wealth,

11:39 spending power measured in dollars per person per day,

11:42 uh,

11:42 calibrated to the international purchasing power parity metrics,

11:46 uh,

11:47 from studies by the World Bank and,

11:48 and others,

11:49 uh,

11:50 and then also details on infrastructure access including electrification.

11:54 Uh,

11:54 built environment,

11:55 access,

11:56 for example,

11:56 access to healthcare,

11:57 schools,

11:58 and so on.

11:59 And so by looking at this at high resolution,

12:01 we're able to triangulate this with the mobility data,

12:03 uh,

12:04 and trace,

12:04 uh,

12:05 the pathways as well.

12:07 Here's an example of uh Google open buildings for Nairobi.

12:11 It's probably one of the highest coverage,

12:13 uh,

12:13 data sets for,

12:14 uh,

12:15 for building footprints in the area.

12:17 Uh,

12:17 and then the OSM road networks which are also well developed,

12:20 uh,

12:21 for this part of Kenya.

12:24 Uh,

12:24 just a quick overview on methods,

12:26 uh,

12:26 combining these data sources,

12:28 so we run through,

12:29 uh,

12:30 some of the typical techniques.

12:31 Here,

12:32 uh,

12:32 we're thinking about being able to,

12:34 to set up these methods in a way that can process data at scale.

12:38 So,

12:38 as I said,

12:39 we're going to go back and look at 3 years' worth of information.

12:42 So our,

12:42 our machine learning pipelines and our,

12:45 our archival uh or uh architectures need to be

12:48 able to consume that amount of information on a,

12:50 on an ongoing basis.

12:51 So if we were to

12:52 expand this to a country scale or to multiple cities

12:55 on the continent,

12:56 for example,

12:57 and what comes out of this is the typical indicators

13:00 that we've looked at in terms of dwell times,

13:02 visit frequencies,

13:03 routing across infrastructure,

13:06 proximity to,

13:07 to various public and private services.

13:11 So what this shows is uh one device and its pathways,

13:16 so one trip,

13:17 one day,

13:17 and one week.

13:19 Uh,

13:20 I've shown it all for one week here for,

13:22 for privacy concerns,

13:23 so that way we aren't able to,

13:25 to pinpoint

13:25 the individual's

13:27 base location or work location.

13:29 Uh,

13:30 we run through the typical steps of

13:32 stop identification to identify the unique locations.

13:36 There's a massive amount of preprocessing

13:38 required here to make sure we're actually

13:40 picking up unique locations as opposed to the

13:44 jitter that's often observed in GPS points.

13:48 We look at a number of destinations of interest,

13:51 so retail and leisure.

13:52 Uh,

13:52 this is a large mall in the Lamington area of Nairobi

13:56 that attracts multiple devices and users

13:59 to that location,

14:00 and then over the course of a week,

14:02 we observed that there are about 200 unique

14:04 visitors from this part of the data sample

14:07 to that location and,

14:08 and those retail facilities of interest.

14:11 So what this helps us understand is not just where

14:13 the people coming from across the urban fabric of Nairobi,

14:16 but in their

14:17 leisure time or their work time,

14:18 where they spend that time and money.

14:22 So I'll switch over to um.

14:31 3 fingers.

14:33 Oops.

14:38 Thank you.

14:39 So,

14:39 for today,

14:40 uh,

14:40 we put together a,

14:41 a demo where we can look at that data interactively.

14:44 So imagine that you're a government entity

14:46 and trying to do road infrastructure planning

14:48 or uh urban or if you're a telecom company

14:51 trying to understand where existing signatures come from,

14:54 but you know that there's a population that hasn't received uh self service yet

14:59 as you're thinking about expanding.

15:00 Your towers or

15:02 a repeater coverage,

15:04 this type of analysis can be useful.

15:07 And so once the safeguards have been built in,

15:09 we're able to then

15:11 showcase

15:12 some of the individual traces as well,

15:15 to be able to pick up information like where the visits are and then be able to

15:20 aggregate to these.

15:22 So,

15:22 uh,

15:23 one of the advantages of planning for this architecture at scale is

15:26 that we're able to deliver the insights directly to a variety of users

15:30 as opposed to,

15:31 uh,

15:31 expecting that every organization that needs this information

15:34 has the capability to,

15:35 to crunch this information on

15:39 on their own or implement the research methods in

15:40 the way that we as researchers would do.

15:44 So,

15:44 uh,

15:45 I'm probably out of time,

15:46 so I'll stop there.

15:47 Um,

15:48 uh,

15:48 one of the main sort of policy implications that

15:51 we haven't touched on today that I'll speak to

15:53 is the idea of procurement.

15:55 For example,

15:56 during COVID,

15:56 we often receive calls from the bank

15:58 on can we use this type of information to continue

16:01 to support bank operations,

16:03 uh,

16:03 and also from ministries who were focused on other

16:06 areas of public need like access to healthcare.

16:09 To be or for example,

16:11 to,

16:11 to help deliver

16:12 uh COVID vaccines in relation to where they were observing activity of people,

16:17 uh,

16:18 and so on in triangulation with,

16:20 uh,

16:20 population density,

16:21 spending patterns and asset wealth.

16:23 So there are a number of applications there,

16:25 um,

16:26 that,

16:26 uh,

16:26 aren't sort of causal inference oriented but,

16:29 uh,

16:29 more designed for immediate services.

16:31 Thank you.

16:39 Thank you,

16:39 Vivek.

16:40 Um,

16:40 our next speaker is Barbara Kramer from IPRI.

16:48 Yeah,

16:48 thank you so much for having me.

16:49 Um,

16:50 also,

16:51 um,

16:52 it's just been really inspiring to see so many great talks today,

16:55 um,

16:55 and I think this is a good time for me

16:58 to put in a plug for the journal where I'm,

17:01 uh,

17:01 editor of the Social Science section.

17:03 Which is uh development engineering.

17:05 and development engineering,

17:07 um,

17:07 publishes work um on rigorous research,

17:11 on innovations to address global poverty

17:14 problems or global development problems.

17:17 Um,

17:17 and then that includes a lot of the mobile data work that has been presented today.

17:21 Both mobile data in terms of strengthening policies and programs,

17:24 but also using mobile data to track

17:27 impacts um of different interventions or to track

17:30 um outcomes and.

17:32 And so in that light also,

17:33 it's good to mention that we currently have a special issue out

17:36 on tracking progress towards the sustainable development goals

17:41 um and if anybody is interested in submitting a paper um do talk to me

17:46 or to Sam Fishman,

17:47 um,

17:48 who is also part of the editorial team and and keeps us as editors uh on track.

17:54 Um,

17:54 so having said that,

17:55 let me now turn my eyes on the ground and talk about,

17:59 uh,

17:59 crop monitoring.

18:01 Uh,

18:01 through smartphone pictures of plots.

18:03 Um,

18:03 this is,

18:03 uh,

18:04 a project,

18:05 um,

18:05 that is joint work with Koon Hufkins,

18:08 who is with Ghent University.

18:10 Um,

18:10 Lillian Weissaka and Benson Juguna,

18:13 who are with Acre Africa,

18:15 um,

18:15 the implementing partner for this work,

18:17 um,

18:17 Acre Africa is an insurance broker and does a lot of work around,

18:21 uh,

18:22 smartphone-based data collection.

18:24 Um,

18:24 and then we're working with,

18:25 um,

18:25 Dwara e-registry,

18:27 which is,

18:27 um,

18:28 an Ag FinTech startup in India,

18:31 um,

18:31 and the Kenyan Agricultural Livestock Research Organization.

18:36 Um,

18:36 so why crop monitoring?

18:38 Why are we talking about,

18:39 you know,

18:39 getting our eyes on the ground on farms?

18:42 Um,

18:42 smallholder farmers,

18:43 they depend

18:44 on,

18:45 uh,

18:45 agriculture for their subsistence,

18:47 and in developing countries,

18:48 a lot of agriculture is rain fed and thereby is exposed,

18:52 uh,

18:53 tremendously to weather risks.

18:55 Um,

18:56 and,

18:56 and this includes droughts,

18:58 floods,

18:58 also,

18:59 you could think of pests and diseases.

19:01 Um,

19:02 and,

19:02 and monitoring this,

19:03 it's just really,

19:04 really important

19:05 to help improve early warning systems,

19:07 um,

19:08 to,

19:09 to provide agricultural advisory services that would allow farmers

19:12 to take preventive measures to minimize damage

19:16 from these extreme weather events,

19:17 um,

19:18 but also financial products such as credit

19:20 or crop insurance to help improve adaptation,

19:23 uh,

19:24 to and resilience to climate change.

19:26 Um,

19:27 now,

19:28 advances in remote sensing have been enormous in the last few years

19:33 to really improve monitoring of crop phonology and,

19:36 and productivity,

19:38 but there's still a number of challenges that are remaining,

19:41 including some technical aspects.

19:43 For instance,

19:43 cloud cover in a rainy season,

19:46 um,

19:46 is is is limiting the accuracy of some

19:48 of the satellite-based algorithms to predict productivity.

19:53 Then there's,

19:53 um,

19:54 resolution issues.

19:55 For small plots,

19:56 there's intercropping in the sense that in a lot of smallholder farming settings

20:01 and as a measure to adapt to climate change,

20:04 we see that farmers are growing,

20:05 for instance,

20:06 beans and maize

20:08 in one plot and how to make sense of that when you're using

20:11 satellite remote sensing,

20:13 how to estimate productivity in those cases.

20:16 Then there's issues that are more social.

20:19 People don't always trust what's being read from satellite images.

20:23 It's not very tangible

20:25 and so understanding becomes an issue

20:28 as well.

20:28 And finally,

20:29 on the empirical side,

20:30 there's a lack of ground truth data

20:32 and a lot of models that have been developed,

20:35 um and a lot of validation studies that have been

20:37 done are focusing on major cereals such as maize,

20:40 wheat or

20:42 uh rice,

20:42 but not on

20:44 other crops that are very important for dietary diversity

20:48 um and for vulnerable populations and more indigenous crops.

20:53 So,

20:53 with that in mind,

20:54 um,

20:54 we started a project in 2016,

20:57 um,

20:58 that is using smartphones

21:00 to,

21:01 we call it smartphone-based near surface remote sensing.

21:04 Um,

21:05 that is using smartphone pictures to monitor crops,

21:07 and it works as follows.

21:09 A farmer can download an application

21:11 on his or her phone,

21:13 uh,

21:13 register a site using an initial picture,

21:16 and then over the course of the season,

21:19 take repeat pictures of that same site,

21:21 and there's visual aids in the smartphone application

21:23 that help tracking this.

21:24 There's also GPS coordinates and timestamps being added,

21:28 so that when the photos are uploaded to the server,

21:31 um this information is all available.

21:33 And then in the application that we're focusing on crop insurance,

21:36 there's actually experts that are reviewing the images and

21:39 indicating whether this farmer has suffered damage and should receive

21:42 an insurance payout.

21:44 And we did a formative evaluation of this in India

21:47 whereby we found that indeed it is possible to track

21:51 uh or to to do crop monitoring through this approach,

21:54 especially to identify crop growth stages,

21:57 um,

21:57 which is useful in the case of advisories

22:00 and to um.

22:02 Uh,

22:02 to,

22:03 um,

22:03 uh,

22:04 identify severe damage which is useful in the case of insurance,

22:08 and then the question became can we do this in

22:09 other settings and so we started working on this in,

22:12 in Kenya,

22:13 um,

22:13 and can we further automate image processing.

22:16 And so in that light,

22:17 um we did a project at a very large scale collecting images

22:23 in Kenya um

22:25 through

22:26 in Kenya where smartphone ownership is a lot a lot lower

22:29 than in the setting in India where we were working,

22:32 um,

22:32 we worked through champion Farmers,

22:34 um,

22:34 which are local opinion.

22:35 Shapers who were equipped by Acre Africa with a smartphone and trained

22:40 on how to send in these images

22:42 and they registered a network of more than 100 farmers per champion

22:47 and out of those we had 20 farmers for whom they took regular photos

22:52 um over the course of the season

22:55 for 3 seasons in a row.

22:57 And all these images were labeled,

22:59 um,

23:00 and in terms of crop growth stage,

23:02 type and extent of damage.

23:04 So,

23:05 um,

23:06 that resulted in a data set of more than 13,000 maize images,

23:10 uh,

23:10 for the three seasons combined.

23:12 Those have all been labeled,

23:14 um,

23:14 and

23:16 out of those,

23:16 we have more than 11,000

23:19 images.

23:20 That are useful that are out there actually

23:23 on the web

23:24 um the images were also used to train

23:26 convolutional neural networks um because so using computer vision

23:30 to detect the labels that we had

23:33 um uh created

23:35 or that the agronomists had been assigning to the images

23:39 um and and other seasons are being added over time.

23:42 So,

23:42 the reason that I was presenting this is that I think this is a great

23:46 data set for those who are interested in agriculture AI and who have remote sensing

23:52 um

23:53 applications who are looking for Ground truth data sets.

23:56 Um,

23:57 this data is out there on the web.

23:59 Um,

23:59 I have a web a website indicated here.

24:02 Um,

24:03 and so also if you have further questions about the data set,

24:06 uh,

24:07 do feel free to get in touch,

24:09 um,

24:09 because we're happy also to share,

24:11 for instance,

24:12 a smartphone application if people want to collect this type of data

24:15 in other locations.

24:16 So,

24:17 that's it.

24:17 Thank you so much.

24:24 Thank you so much Berber.

24:25 It seems a recurring theme is that taking pictures

24:29 of stuff with your cell phone is really great,

24:31 whether it be bacteria or

24:34 worms.

24:34 Sorry,

24:35 I can't,

24:36 I don't remember exactly which.

24:37 Sorry,

24:37 Maria,

24:38 um,

24:38 OK,

24:38 so our,

24:39 our next,

24:39 uh,

24:40 presenter is Sarraj Nair.

24:41 Uh,

24:41 he's been mentioned a couple times in the conference and now here is the man himself,

24:46 not to put you on the spot.

24:47 Thank you.

24:50 Um,

24:52 Oh,

24:52 awesome,

24:52 thanks.

24:53 Um,

24:54 all right.

24:55 Good evening,

24:56 everyone.

24:56 I guess we're almost done.

24:57 Wine,

24:58 woo-hoo.

24:59 Um,

25:00 yeah,

25:00 I'm excited to talk to you today about,

25:02 um,

25:02 ongoing work with Josh Blumenstock from UC Berkeley.

25:05 Um,

25:05 we've been looking at the impact of cash transfers on

25:07 mobile money usage and then more broadly digital financial inclusion.

25:11 Um,

25:12 in terms of motivation,

25:13 I guess we've heard several times today that in the last couple of years,

25:17 there's been a massive digitization of social protection.

25:20 Um,

25:21 I think this is the last accurate number.

25:23 I have over 150 programs delivered aid

25:26 via digital channels during the COVID-19 pandemic,

25:29 um,

25:30 and a large number of these programs relied on mobile money to deliver this aid,

25:34 especially in,

25:34 um,

25:34 sub-Saharan Africa.

25:37 Our paper focuses on this idea that

25:40 delivering aid via mobile money or mobile money transfers

25:42 could serve as an on-ramp to broader financial inclusion

25:46 by providing access to very basic financial services for low-income individuals

25:51 who are often excluded from the traditional banking system.

25:54 So,

25:55 this paper very specifically simply looks at this question

25:57 of whether short-term exposure to these digital cash transfers can

26:01 drive the longer-term usage of the broader mobile money ecosystem.

26:05 I guess it complements this very

26:07 quickly growing set of papers that generally look

26:10 at the impacts of mobile money transfers.

26:12 Most of those papers focus on

26:14 welfare outcomes such as consumption or

26:16 women's economic empowerment or intra-household bargaining

26:19 or on efficiency gains for firms and employers.

26:22 Much of this literature

26:24 finds that short term exposure to

26:25 cash transfers doesn't really change financial behavior

26:28 in the long term.

26:29 Um,

26:30 at least in the past,

26:31 pre-pandemic.

26:31 We think this is going to change after the

26:34 evidence from the COVID-19 studies come out.

26:38 Um,

26:38 the context we're focusing on is that of the OBC program in Togo.

26:42 Um,

26:42 Emily spoke at this,

26:44 uh,

26:44 spoke about this in the morning at length,

26:46 so I'm not gonna spend too much time on this.

26:47 I just wanna mention that we focus on a phase of the program.

26:51 Um,

26:51 that,

26:51 that had a randomized rollout.

26:53 So a random set of individuals were assigned to a treatment group

26:56 who began receiving cash transfers in November 2020 for a five-month period,

27:00 following which the control group received transfers.

27:02 So we leveraged this randomization to study these impacts on mobile money usage

27:07 that I'm going to talk about.

27:08 Um.

27:10 Yeah,

27:10 that's the only thing I wanted to mention here.

27:12 Um,

27:12 I won't spend any more time.

27:14 We,

27:14 we use data from uh one of the largest telecom operators in Togo.

27:18 Uh,

27:18 this is a rich data set that

27:20 gives us information on when individuals sort of

27:23 withdraw and deposit money into their mobile money accounts,

27:25 but it also gives us rich insight into their saving behavior,

27:28 into the network of financial transfers,

27:30 and how they're using the broader mobile money

27:32 ecosystem to make digital payments and purchase airtime

27:35 and to pay for things like cable and water and electricity and things like that.

27:40 Um,

27:41 yeah.

27:42 So before I jump into the main results,

27:43 I think

27:44 my,

27:44 the thing I really like about the analysis so far has

27:47 been that simple descriptive stuff provides us a lot of insight.

27:51 So,

27:51 I'd like you to focus on this panel on the top where I'm simply plotting out

27:55 the fraction of individuals making at least one

27:58 mobile money transaction in a given month.

28:00 Remember that the transfers we're interested in began in November 2020.

28:05 So,

28:05 prior to that,

28:06 basically,

28:07 roughly 20% of our sample was making a mobile money transaction.

28:11 The moment people start receiving these transfers,

28:13 you see this huge spike in activity in their mobile money accounts which

28:17 um

28:18 kind of sustains for the five months during which they

28:20 receive transfers and then there's a decline of sorts.

28:23 The interesting thing here,

28:24 um,

28:24 and I think this is what differentiates these results from,

28:26 I guess,

28:27 stuff that's out there in the past is Panel B where

28:30 we're able to show that

28:31 A lot of this increase in mobile money usage is actually driven by

28:36 an increase in the use of wider ecosystem

28:38 services such as bill payments and transfers and

28:41 other things,

28:42 so people are not simply using their accounts to withdraw cash

28:45 withdraw cash and then spend it in cash locally.

28:47 There's actual ecosystem-wide transactions happening in this setting at least.

28:51 Right?

28:51 And so this sets up

28:53 sort of my

28:54 one results side.

28:55 I promise not to show tables,

28:57 so

28:58 it's all text.

28:58 Um,

28:59 but yeah,

28:59 so basically,

29:00 what happens when individuals receive transfers in a month,

29:02 they receive transfer,

29:03 we see this like

29:04 62% increase in the

29:06 usage of their mobile money accounts relative to baseline levels,

29:09 that's a 300% increase.

29:11 Um,

29:12 the nice thing,

29:12 the encouraging thing rather is that these increases

29:15 are ecosystem-wide,

29:17 so there's people actually making digital transactions during

29:20 this period.

29:20 Um,

29:21 what's,

29:22 what we are trying to look at and get a

29:24 better sense of is what happens in the longer term.

29:26 So after the cash transfers conclude,

29:29 do these gains persist or does everyone go back to using cash?

29:32 And I think at least our initial results

29:34 suggest that there are modest gains in mobile money

29:37 activity,

29:37 mobile money account usage in the longer term.

29:40 Our data

29:41 lasts for about 4 months after the cash transfer program ends.

29:44 What I'm showing you here are results from an event

29:46 study analysis where we're comparing individuals who received a cash transfer

29:50 to individuals who never received one.

29:53 So,

29:53 relative to baseline levels of

29:56 mobile money usage,

29:56 these results show the same thing.

29:58 There's this large initial increase in the months

30:00 during which they receive a cash transfer,

30:02 and then there's a drop-off of sorts in mobile money account activity,

30:05 but

30:07 Even 4 months after the cash transfers conclude,

30:10 individuals who receive transfers remain much more likely to use mobile money than

30:15 individuals who don't receive these transfers.

30:17 So,

30:17 that's the broad pattern we see.

30:19 Um,

30:19 we think there are some specific,

30:21 um,

30:21 factors in Togo that contribute to these increases.

30:24 Um,

30:24 Togo,

30:25 they subsidized the transaction costs to withdraw money,

30:28 at least initially.

30:29 Uh,

30:29 we think this helped.

30:30 There's a dense agent network and the Togo government was

30:32 also running these financial literacy programs even before the pandemic,

30:36 all of which could have helped familiarize people with some of these aspects and

30:40 um.

30:42 Lead to the results that we're seeing.

30:43 What are the implications of this?

30:45 We think the NoviC program,

30:46 based on rough back of the envelope calculations,

30:49 added

30:50 at least 80,000 customers to the ecosystem in the long term.

30:53 Um,

30:54 and again,

30:54 using like reference numbers available from exercises that

30:57 have been done in West Africa and Kenya,

30:59 we think

31:00 this means up to 80 million

31:02 um Togolese francs in additional revenue and potentially up

31:05 to 160 million in terms of true economic value.

31:08 Uh,

31:08 added to the economy.

31:09 But

31:10 these are back of the envelope calculations,

31:12 we're trying to be better about this,

31:13 um,

31:14 but yeah,

31:15 that's my results.

31:16 Um,

31:17 thank you.

31:18 Feedback is welcome.

31:19 This is the first time we're presenting these results,

31:20 so we would love to hear your thoughts.

31:23 Thank you.

31:30 Thank you so much,

31:31 Sarraj.

31:32 Um,

31:32 next up we have,

31:33 uh,

31:34 Winnie Mugogu from Busara.

31:36 Uh,

31:36 we're really excited to have her and thank her for,

31:38 thank her for coming all the way to Kenya to pre from Kenya to,

31:42 uh,

31:42 to present,

31:43 uh,

31:43 these,

31:44 uh,

31:45 results,

31:45 and I'm hoping that while I was talking,

31:49 Napur would come to change the slides,

31:50 but that has not happened.

31:51 All right.

31:57 Oh,

31:57 there we go.

31:58 Cool.

32:04 Um,

32:04 did it all by myself.

32:05 All right.

32:06 All right,

32:06 Winnie,

32:06 the floor is yours.

32:12 OK.

32:12 Hello,

32:13 everybody.

32:13 Um,

32:14 today I'll be presenting on um moving from

32:16 in-person to mobile data collection in Kenya.

32:19 I should have said at Busara,

32:21 but the at Busara is silent.

32:23 Um,

32:24 I'm a research specialist at Busara Center for Behavioral Economics,

32:27 which is in Nairobi,

32:28 Kenya.

32:29 So I'll just give a background on Busara.

32:32 Um,

32:33 methods,

32:33 research agenda that we have going on and

32:35 some studies that use mobile data collection.

32:38 So I'll highlight a lot on the lessons learned from these studies

32:41 more than the specific results.

32:43 If anybody wants,

32:44 is interested in any of the studies and the specific results,

32:47 they can consult me on that.

32:49 So,

32:50 background.

32:50 Um,

32:51 Basically,

32:52 BUSARA Center um is a research institution

32:55 and a consultant um they do consulting work

32:58 with researchers and organizations to advance and apply

33:01 behavioral science in pursuit of poverty alleviation.

33:03 That's the mission of BUSARA.

33:05 So basically,

33:06 we have a nice

33:07 behavioral lab in Nairobi,

33:09 Kenya.

33:10 Possibly the only one in Africa,

33:12 but I could be wrong,

33:13 so

33:14 anyone can correct me on that one.

33:16 but we um conduct some behavioral studies in Nairobi,

33:19 Kenya.

33:20 In March 2020,

33:21 I moved from my home country,

33:23 which is Malawi

33:24 to Kenya.

33:25 And I arrived in Kenya and a few days later,

33:29 there was the first COVID case.

33:30 A few days later,

33:32 they said we're not going to work next week.

33:34 So I never actually got to see this lab and I never got to experience um

33:38 all the wonderful experiments that people carry out.

33:41 But as a research institution,

33:43 of course,

33:44 we had to somehow carry out the research.

33:46 We needed Data.

33:47 Um,

33:48 as such,

33:48 yeah,

33:49 the pandemic,

33:50 it came in the way of um

33:51 data,

33:52 in-person data collection

33:53 as most of the studies today have been motivated by,

33:55 you know,

33:55 the pandemic

33:57 and so

33:58 we had to think of how do we continue to collect data,

34:00 otherwise,

34:00 we can't carry out our operations.

34:03 And so

34:04 we decided,

34:04 well,

34:05 remote or mobile data collection

34:07 and this is in a low-income country setting um

34:11 We had the subject pool of respondents in

34:14 Kibera and Kwangware,

34:15 which are

34:17 um urban slums in Nairobi,

34:18 Kenya.

34:19 If you see a lot on slums in the news,

34:21 it's probably in Kibera,

34:22 very um the largest urban slum in Africa,

34:25 actually.

34:26 Um,

34:27 and so as we were thinking around how do we continue um

34:31 data collection,

34:32 that's when we came up with this methods research agenda.

34:35 So I had the methods research agenda at BASARA

34:38 and basically,

34:39 we are,

34:40 we have a few questions that we're trying to answer.

34:42 I've just selected two of the most prominent ones

34:44 um that we're trying to answer with this methods research agenda.

34:48 It's how we can contextualize,

34:50 measure

34:51 and improve access.

34:53 Response,

34:54 attention,

34:54 comprehension

34:56 of surveys

34:57 and experiments for populations in the global South.

34:59 And this is going to be specifically be applied to remote

35:03 data collection because that's what happened during the COVID pandemic.

35:06 We couldn't do in-person

35:07 data collection.

35:08 So,

35:08 we're looking at,

35:09 you know,

35:10 still improving access,

35:12 still getting good quality data,

35:14 still getting um

35:16 you know,

35:16 good attention even though the,

35:17 the

35:19 research is being carried out remotely.

35:21 And then how do you,

35:22 how do the answers to these questions vary across gender,

35:25 racial,

35:25 national,

35:25 and economic groups?

35:26 I think a lot of the discussion today as well has been,

35:29 you know,

35:29 around

35:30 um do we see differences in gender when we're using mobile data collection.

35:34 These are some of the questions that we're going to be answering

35:36 and some of the studies that I'm going to share.

35:39 Um,

35:40 begin to answer those questions.

35:42 It's going to be carried out over the next 3 years.

35:44 If anybody's interested in collaborating to answer these questions,

35:47 um feel free to do so.

35:49 So,

35:50 One of the ways we innovated was through

35:52 what we called Kite.

35:54 Kite is um which sought for Knowledge and Insights tool for experiments.

35:59 So basically,

35:59 we wanted to continue doing this behavioral econ um experiments

36:03 but on,

36:04 on a mobile app.

36:05 Um,

36:06 so

36:07 It was created in-house in Bossara

36:09 and one of the studies that we um

36:12 we did on

36:14 the app was a financial safety nets and attention study.

36:17 By that,

36:17 it was

36:18 how are people coping over the

36:21 COVID pandemic financially.

36:22 I think these are important questions to ask at a time where

36:24 we couldn't visit the people in uh like we couldn't go in person

36:28 but we wanted to know how they're doing financially.

36:30 Um,

36:31 and then we added in this part of attention.

36:34 Now,

36:34 attention is because

36:35 when you're thinking of remote data collection,

36:37 you don't have the enumerator or

36:40 the FO to the field officers to

36:42 keep people's attention going.

36:44 So how are we so sure that the um with remote data,

36:46 people aren't just clicking away at answers and then submitting?

36:49 I'm sure we've done that sometimes when you get an email

36:52 asking you to complete a survey and just kinda click,

36:54 click,

36:54 click submit.

36:55 Um,

36:56 and so

36:57 we try to Answer

36:59 questions like

37:00 the sources of income people used before and after the pandemic,

37:04 but also the attention

37:05 part would be would ask questions like,

37:07 were you born before 1940.

37:09 The inclusion was people who were a specific age and

37:11 there was no way they should be born before 1940.

37:14 That's how we would measure something like attention.

37:17 So,

37:18 we ran this with um a target sample of 500 people but you only reached 375.

37:23 Um

37:24 getting people to download the app was quite a challenge um and so

37:28 we're actually pivoting

37:30 how we're going to use Kite so that we include a lot of people

37:33 um

37:35 given how difficult it was to get people to download it.

37:38 Um,

37:39 this is a study of,

37:40 uh,

37:40 chlorine,

37:41 uh,

37:41 the study's actually called Can Simple Psychological

37:43 Interventions Increase Investment in Preventative Health.

37:46 I've included this study but it's by Kate Orkin and Annette John.

37:49 Kate Orkins at Oxford.

37:51 Um,

37:51 collaborated with Bosara.

37:53 When I'd arrived,

37:54 we were supposed to do a follow-up survey

37:57 in person.

37:58 We had to pivot to,

38:00 um,

38:00 a phone survey.

38:02 We managed to reach

38:03 3,194 women in rural

38:06 Kenya,

38:06 in

38:07 Bungoma and Kakamega.

38:09 And to reduce attrition,

38:10 we went and tracked respondents on the ground,

38:14 but they still did the surveys over the phone

38:17 and for those who didn't have a phone,

38:19 the people who did the tracking,

38:20 they borrowed the respondent the phone so that

38:22 they can still take part in the survey.

38:24 So that's one of the ways in which you try to

38:25 include people who would otherwise have not been in the survey.

38:28 Um,

38:29 I'll skip this,

38:30 uh,

38:30 study on we use SMSs because there's the one study

38:33 which we found very similar results that was just presented,

38:36 um,

38:37 and cause of time.

38:38 Um.

38:40 And then we've also tried um

38:43 to do call tricks and

38:45 compare that to phone surveys.

38:46 I'd actually initially wanted to present on this study

38:49 on using the LT experiment,

38:50 which is an indirect method of measuring intimate partner violence,

38:55 um comparing it over result um findings on court tricks and

38:59 on a phone.

39:00 It has taken quite some time to start data collection because of IRBs and you know,

39:04 the sensitive issue of IPV

39:06 but we finally got an approval.

39:07 We're starting data collection this week

39:09 if anybody's interested to find out.

39:11 Um,

39:12 just the idea behind this um study with the list experiment is that

39:16 um with intimate partner violence,

39:18 with sensitive topic.

39:19 topics,

39:19 people aren't so open to share about their experiences

39:23 and so the idea with this methods research agenda is that we

39:26 want to make sure that we are collecting good quality data.

39:30 And so

39:31 today,

39:31 the idea is how do these things,

39:33 how do these things affect policy?

39:35 What,

39:35 uh what we're trying to get from these studies is basically that

39:38 Policy must be guided by evidence

39:41 and can we trust the evidence?

39:43 We can only trust the evidence if they come from good methods

39:46 and that's why we looked at things like this.

39:49 So um that's why I had to share.

39:52 Like I said,

39:52 there's so much I could have shared

39:54 specifics of the study,

39:55 but

39:56 yeah,

39:56 I just didn't have time for it.

39:57 Um

39:58 thank you.

40:05 That's a great quote.

40:07 Uh,

40:07 you can only trust the evidence if it comes from good methods.

40:10 Um,

40:10 so our last speaker of the day,

40:12 uh,

40:12 is gonna be,

40:13 uh,

40:13 Zach White from GSMA,

40:15 which,

40:15 as we've seen in,

40:16 in,

40:16 in several of the presentations of prior speakers,

40:18 you know,

40:19 really makes possible a lot of,

40:20 uh,

40:20 a lot of the research.

40:21 So,

40:21 um.

40:23 Yes.

40:24 All right,

40:24 great.

40:24 Thank you.

40:34 This is it.

40:35 That's it.

40:43 Fuck.

40:44 Uh,

40:45 so I really am standing in the way

40:47 of the last thing.

40:49 But I'm gonna do 3 things today which is just introduce the GSMA and

40:52 then move on to 3 frameworks that we use to look at partnerships,

40:57 uh,

40:57 and particularly around mobile big data.

41:00 And then I'm gonna talk through some of the cases

41:03 where the GSMA played a very active role in facilitating those

41:07 in the context of the COVID-19 response.

41:10 And you can count them down as 321,

41:13 should you choose to.

41:15 Uh,

41:15 so the GSMA and talking to people it seems about 50/50,

41:18 uh,

41:18 those who know us,

41:19 those who don't.

41:20 It's the industry association for the mobile industry,

41:23 so our members are about

41:25 800 of all of the major mobile network operators and associated ecosystem players.

41:31 Uh,

41:32 we convene the industry,

41:33 there's industry services and events,

41:35 but then also within the mobile,

41:37 uh,

41:38 the GSMA is the Mobile for Development Foundation

41:41 that sits at the interface of development and the industry.

41:45 And I am one of,

41:46 I work with the Digital Utilities Program which is

41:48 one of 9 kind of thematic programs working across,

41:52 you know,

41:52 agriculture,

41:53 mobile money.

41:55 Uh,

41:56 uh,

41:56 and many others that

41:58 my brain can't recall recall right now.

42:00 Uh,

42:00 and within the utilities program we focus on energy,

42:03 water,

42:03 transport,

42:04 sanitation,

42:05 and waste management,

42:05 those services.

42:07 And what we do in,

42:08 this is similar across all of the

42:11 programs and portfolios is most importantly,

42:14 one is we give innovation grants uh to predominantly startups and SMEs.

42:18 To date we've given about 100 grants

42:20 totaling nearly 20 million to about 100

42:24 organizations.

42:25 Facilitating with those about 80 partnerships with MNO's and I

42:29 think the kind of headline numbers are within 2 years,

42:32 those organizations went on to raise over half a billion,

42:35 something like 530 million

42:37 and impact 36 million people.

42:39 Alongside that we do research and insights and uh convening uh.

42:44 For the industry.

42:46 So

42:47 that's the first bit down.

42:48 Number 2 is the frameworks we use to think about uh partnerships.

42:53 And this is the first one which is also,

42:55 it's a process but it's also the different stakeholders

42:58 and it's also the technical aspects that come in.

43:00 So starting with the MNO data,

43:03 it distinguishing between the different types of data that come

43:05 through and being clear about which you need and then

43:08 the process that goes into making those useful within the warehouse.

43:12 Secondly,

43:13 where it comes in

43:14 the third party data and how we've talked

43:16 about combining those and the specific analytics you need

43:19 to get those useful

43:20 uh packages out for the decision makers.

43:23 And of course these can be different people across different stages.

43:26 Uh,

43:27 so the journey map for partnership describes

43:29 those elements that are particularly important,

43:31 and I think

43:32 a lot of people will be familiar here with the partnership establishment

43:36 of how you get to that first stage of getting the data

43:39 and then similarly,

43:40 the issues with the data management and the integration.

43:43 But I think what

43:44 maybe the research community doesn't think so much about is then

43:48 the sustainability of these,

43:49 the financial sustainability.

43:51 And then also ultimately

43:53 the impact and usage and how it all comes and

43:56 the fact is you can move between these different elements or

43:58 look at them in a different way as different starting points.

44:02 Um,

44:03 I'll skip part this this very quickly,

44:05 but we use the haves and needs framework quite a lot in partnership establishment

44:09 to really think about where the value's coming from on both sides and to be

44:13 clear

44:14 about that from the start.

44:18 That's frameworks down,

44:19 23,

44:20 COVID response is uh led by our AI for

44:24 Impact initiative which sits outside of Mobile for Development.

44:27 But this is a task force of 21 mobile operators covering 2

44:32 billion people

44:33 um

44:35 that provide that has an advisory panel and and

44:37 works to develop specific initiatives around mobile big data

44:41 and AI,

44:43 um.

44:44 We've seen this

44:45 already with fair to slide,

44:47 and as mentioned,

44:47 I mean

44:48 the big spaces of white where not a lot of work went on.

44:52 Uh.

44:54 So the GSMA worked

44:55 with support from the UK Foreign and Commonwealth and Development Office FCDO

45:00 uh in about 14 countries,

45:03 providing there some advice,

45:05 some technical advice,

45:06 some consulting on

45:08 ethics and and business practice,

45:09 but

45:10 in 4 countries,

45:11 Benin,

45:12 Burkina Faso,

45:13 DRC and Rwanda,

45:14 we worked more intensively

45:16 to facilitate some public-private partnerships and,

45:19 and they're what I'll talk about in a minute.

45:21 Um,

45:22 so I think this is also very familiar to,

45:25 to what we've been talking about today,

45:27 but going back to that first slide and thinking about.

45:31 You know,

45:31 during the process what could have been uh informing these partnerships,

45:35 where different data and particularly from different agencies needed to come in

45:39 to produce different sorts of analytics for different sorts of decisions.

45:43 And this was really

45:44 uh some of the thinking that was

45:48 done at the very start of the process to

45:49 determine what happened in each of those four countries.

45:53 Uh,

45:53 and it's slightly out of place but that's the output for

45:56 DRC in terms of the dashboard on kind of mobility.

46:00 Um,

46:00 as well as,

46:02 um,

46:02 some other analytics.

46:04 Uh,

46:05 so this is revisiting part of that but just to kind of package,

46:09 uh,

46:09 exactly the pro the process that the AI for

46:12 Impact team went through in those four countries.

46:16 Um,

46:16 I'm gonna skip past it to,

46:18 to some of what came out.

46:20 Oh,

46:20 that hasn't scaled very well.

46:22 Um,

46:23 but,

46:24 uh,

46:24 I think,

46:25 you know.

46:26 It was,

46:27 there's 6 things that that we point to that we were proud of within this work.

46:31 I think the first is that you know it was a

46:32 successful use case development for mobile big data products and to

46:37 develop and deliver those in 4 countries

46:40 producing data for decisions in a very short time frame

46:43 uh was an enormous achievement.

46:45 Uh,

46:45 it increased awareness among some of those key stakeholders.

46:48 Many of them were learning that interacting with

46:50 these products or concepts for the first time.

46:52 It promoted a state of privacy and governance,

46:55 so kickstarting a lot of discussions that

46:58 that

46:59 you know perhaps hadn't been considered before,

47:00 especially through the the government agencies that were worked.

47:04 Uh,

47:04 there was a lot of kind of enhancing of technical skills of those key,

47:07 key actors,

47:08 um,

47:09 hopefully laying some of the foundations for,

47:12 for the ecosystem growth,

47:13 uh,

47:14 and,

47:14 and then also our role in,

47:16 in efforts to increase knowledge sharing around this.

47:20 Um

47:22 This maybe you can revisit the slides in your own time cos I'm I'm running out of time.

47:26 er but this this draws on some of the other work

47:29 that we've done around innovative data for urban planning and points to

47:33 you know the checklist of things that that really

47:35 should be considered in in partnership formation and also

47:38 could present stumbling blocks at different times.

47:41 Uh,

47:42 the penultimate slide is just some of the resources that you may

47:46 want to visit and it's it's kind of more in there for.

47:48 Uh,

47:49 revisiting at a later date,

47:50 but to,

47:50 to flag some of the key resources on ethics and and principles for using,

47:55 uh,

47:56 big data AI.

47:57 Importantly,

47:58 and something that hasn't been talked about a lot

47:59 apart from in this lightning talk session is,

48:02 is the business models that go with that.

48:04 Uh,

48:05 especially if you want to,

48:06 to have sustained use and you know where third parties

48:09 come in to apply these analytics to to produce useful products and services.

48:13 Um,

48:14 and

48:15 then there's also some of the utility specific resources there,

48:19 and I think with that it

48:22 ticks off the final point for the day and thank you very much.

48:32 Um,

48:33 all right,

48:33 well,

48:34 thank you so much.

48:35 Um,

48:36 I have a couple of closing

48:38 remarks.

48:39 Um,

48:40 so first of all,

48:41 I wanted to thank you all for being here and it's so,

48:45 um,

48:45 exciting and it's so valuable to be here in person

48:49 and it has been like truly a different

48:52 experience for me because I've only been to maybe

48:54 two conferences in person.

48:57 And the amount of that I've gotten out of today is just

49:00 astronomically different than what I would have gotten out of it,

49:03 you know,

49:04 virtually or in another situation.

49:05 So I hope that that's the case for you and um,

49:09 yeah,

49:09 thank you so much for,

49:10 for,

49:10 for coming here to the World Bank.

49:12 Um,

49:13 another benefit of in person is that we all get to like chat,

49:16 uh,

49:17 and not in a breakout group.

49:18 So,

49:19 um,

49:19 I hope that any questions that you have for

49:22 some of the speakers for the lightning talks,

49:23 for example,

49:24 you're able to ask them,

49:26 uh,

49:26 during the reception.

49:28 Um,

49:29 I just wanna make a note for Berber.

49:31 There was a specific question for you.

49:33 Oh,

49:33 addressed,

49:34 perfect.

49:34 OK,

49:35 great,

49:35 um.

49:37 Next,

49:37 I really want to thank,

49:38 uh,

49:39 some of the co-organizers of this event.

49:41 So some of them are standing right here,

49:43 um,

49:43 they've been

49:44 wanna come out and

49:45 and give a little wave or something,

49:47 um,

49:48 so these are,

49:48 uh,

49:49 the people who helped organize this event and have kept it running,

49:52 and it is truly like,

49:53 uh,

49:54 a large effort on the back end.

49:55 So let's give a round of applause for.

showAllTimestamps
no
transcript
So we're gonna have 6 lightning talks, uh, uh, to, to follow, uh, session, uh, session 4. I'm gonna MC, uh, the, the lightning talks kinda get things, uh, moving along. Um, each lightning talk, uh, is gonna be 7 minutes, and I'm gonna be mean to you if you go over time, um, but you know, I hope that they're informative, uh, in, in, in a short, uh, in a short amount of time. Um, OK, so we're gonna start with, uh, Marissa Noii from ID. is 42 and I'm gonna let kind of the panelists up here get back to, get back to their seats. And just to quickly note that the reception this evening at 5 will be in the atrium, which is up two floors, the main, um, the main floor of the World Bank ground level. So that's which is exciting I've been told. OK, awesome. Yeah. And if you Alright. Hi everyone. My name is Marissa Nowicki, and I work with Ideas 42. Knowing I'm the only person keeping you from wine right now, I'm gonna go through this quickly, um. Uh, so today, I'm gonna just give you a brief introduction on who we are, who IDEAS 42 is. Um, we're going to talk briefly about three different case studies and projects that we did during COVID, um, and how we pivoted to use mobile data collection very briefly, um, and then some key insights that we learned from this experience. So for those that don't know, Ideas 42 is an independent nonprofit firm. It was originally founded in 2008 by a group of academics that worked in behavioral economics, and they wanted to make sure that their work was applied to real world situations and used for social good. Essentially we study human decision making and behavior. Um, and what we do is first we help define a behavioral problem. We study the context to understand why the desired behavior is or is not occurring. Um, we design a behaviorally informed intervention, we test that solution and then we scale successful interventions. And throughout our methodology, data is crucial. It's important to identify the behavioral barriers, to create effective behaviorally informed interventions, and then to make sure that our designs are working. Um, and during the COVID-19 pandemic, we needed to utilize new and creative data collection methods so that way we could pivot and continue doing our work. So I'm gonna briefly talk about 3 case studies where we pivoted, um, and in these instances, instead of doing our normal in-person activities, we decided to do IVR or interactive voice response, which is essentially when a pre-recorded message, um, is spoken to you and then you click a number which we've all interacted with, you know, uh, for English, please press 1. I think we've all, we've all experienced that and then also telephone interviews. So the first one is from the DRC, um, and in one of our projects, we are working to help beneficiaries of cash transfer programs, um, use this money to build resistance against economic shocks. We knew we needed a really large sample for this to run a fully powered randomized control trial, so about 5000 was our goal, and that that would require a lot of resources. And as this was newer for us, we decided to first do a pilot. Make sure that we could get the reach that we actually wanted, so we ran a pilot IVR survey with about 200 randomly sampled people to assess baseline response rate and if there was anything we could do to improve that experience. The good news was about 67% of people answered the phone and about 20% completed the survey. And a few key insights from this, um, were that, you know, there were some technical difficulties, which is, uh, I think a common theme that we want to be mindful of who can participate, who might have a little bit of difficulty participating. Uh, some interesting notes on the language, so the local language Langala was preferred for speaking and listening, whereas French was preferred for written messages. So having an option to have it in the local language, but then the option to switch to French if they would like. We saw that times of day, um, and the days of the week that we called were really important, so calling during weekend work, weekday afternoons had the best response rate. So understanding when people wanted to be called, what languages were the most important, and then of course offsetting any potential costs, we did this through airtime so that people, uh, did not have to pay anything to participate. So we use this to refine our approach and we did ultimately run the RCT and got the 5000. Um, completed responses, so that was a success that went well. Um, around the same time, we were also working in Kenya, um, and prior to the pandemic, we had been in the middle of working with the Kenyan government to enhance the impact of social protection cash transfers, and then when travel restrictions were in place, our work had to stop. Um, at that time we had been in the process of rolling out a wide scale randomized control trial. We had already done the trial in 3 of the 4 intended counties, um, and the data collection was scheduled to begin a couple months later and so to make the. of the situation, we decided to do something that was a contact-free approach, so using mobile data collection this time through phone interviews, um, and the goal was to assess, among other questions, we also had to pivot our research approach and our questions, um. But the goal was to see if the behavioral interventions that we designed in a pre-COVID context were still useful to those beneficiaries during the pandemic. Uh, and we were able to see using this, uh, using this tool that people who had been included in the interventions were more likely to have concrete savings goals and were more likely to actually be working towards those savings goals. So although it wasn't a full replacement for in-person surveys, it was a useful tool to help us see, uh, that there was an effect. And then in the last project I'm just gonna briefly talk about, we've been working with the World Bank to adapt a behaviorally designed business training program which we call the Financial heuristics training to an audience of female small business owners in Ethiopia. And initially we would, or what we would usually do would be to go to Ethiopia and do contextual reconnaissance, um, some behavioral diagnostics, carry out the interventions in person, but of course we couldn't do that, so we had to adapt, um, and part of what we did was we hired someone locally to conduct interviews over the phone and see, uh, get some baseline data, see how we could refine our approach and how we could. Adopt it, um, and there were lots of lessons learned about this project, about how people appreciated callback numbers and options, um, actually the value of just a step up from IVR but giving them a little personal interaction was really valuable to them. Um, they appreciated it and of course this wasn't again a perfect substitute but was useful for us to collect useful data and then pivot, um, and adjust our approaches. So just I realized that was a lot of information very fast, um, in summary. This is a really good tool to have in your tool belt. Um, I don't think this is a replacement for in-person data collection, but it is a good way to complement data you're already collecting, or You know, in the unprecedented times we're living in, if you need to have a stopgap measure, this is a good way to, um, to achieve what you need to achieve. And then just some simple considerations that we found useful to think about if you're considering using those tools were the sample size, the level of detail needed, and the potential value add of personal interactions. All right, thank you for everyone. Uh, if you have any questions, feel free to check out our website or send me an email. myself. OK, so our next speaker, uh, is Vivek, um, and would you mind pulling up the, the link, link. OK, and this is what you wanted and needed. Yes, uh, I'll start with the slides first though. OK, great. Just on the other. Thank you. So when you wanna switch it, just like slide it. Perfect. All right, Vivek Saranni from Atlas AI. Thank you. Hello everyone, I'm Vivek Sarani, head of applied data science at Atlas AI. Uh, before jumping into some of the technical components of the talk, a quick word on our company. Since, uh, I'm not affiliated with the university or the bank, we are a software engineering company that uses remote sensing, uh, machine learning, and, and ground data sources to build high resolution data sets and then translate that into urban and rural applications for both the public sector as well as the private sector. So in this talk, I'll focus on this idea of where people spending their time and money. It's shown up in many ways today, but the main difference I would say between many of the talks today and this one is that we're particularly focused on urban service design. So how can the public sector or private companies focus on Where to increase capacity or deliver additional services. So probably Alice's talk comes closest to this and thinking about transportation applications in the Lagos context. And so the, the, the easy key question to hold on to in, in this, this idea is where are people spending their time and money. And we're looking at mobility data in the Nairobi context. We're also combining this with high resolution data sets, uh, for example, the asset wealth index, which is an indicator of poverty. In, in Nairobi, this is a proprietary data set that we built ourselves, and then curated sources on urban, urban built environment, for example, Google, Google open buildings, but also other types of scraped information like points of interest data sets. And then a number of the methods which uh researchers in this room are, are familiar with, uh, including safeguards for, for the data and then also some of the optimization techniques for, for modeling. So, uh, I'll skip over some of the contextual slides since, uh, at this point in the day we're all very familiar with some of the challenges and issues, but again, uh, the idea that I'll focus on here in this talk is that we're interested in not just the location properties of mobility. Also the interaction of that mobility information with the built environment, whether it's physical assets, assets like roads and transportation infrastructure or built places, stores, retail environments, homes, or other types of facilities. So, in, in the study, we're looking at 3 years' worth of mobility data for all of Kenya. Uh, but for today, I'm focusing on about 3.5 million observations from 1 week alone. And within the Nairobi envelope, that's about almost 2 million observations from 150,000 unique devices. So all the caveats of sample bias and the, the, the, the limits on who owns smartphones obviously hold in this case as well. Ah, but we're gonna try to dig a little bit deeper into them. Uh, the proprietary Atlas AI data is, um, 1 kilometer by 1 kilometer resolution, uh, indices on asset wealth, spending power measured in dollars per person per day, uh, calibrated to the international purchasing power parity metrics, uh, from studies by the World Bank and, and others, uh, and then also details on infrastructure access including electrification. Uh, built environment, access, for example, access to healthcare, schools, and so on. And so by looking at this at high resolution, we're able to triangulate this with the mobility data, uh, and trace, uh, the pathways as well. Here's an example of uh Google open buildings for Nairobi. It's probably one of the highest coverage, uh, data sets for, uh, for building footprints in the area. Uh, and then the OSM road networks which are also well developed, uh, for this part of Kenya. Uh, just a quick overview on methods, uh, combining these data sources, so we run through, uh, some of the typical techniques. Here, uh, we're thinking about being able to, to set up these methods in a way that can process data at scale. So, as I said, we're going to go back and look at 3 years' worth of information. So our, our machine learning pipelines and our, our archival uh or uh architectures need to be able to consume that amount of information on a, on an ongoing basis. So if we were to expand this to a country scale or to multiple cities on the continent, for example, and what comes out of this is the typical indicators that we've looked at in terms of dwell times, visit frequencies, routing across infrastructure, proximity to, to various public and private services. So what this shows is uh one device and its pathways, so one trip, one day, and one week. Uh, I've shown it all for one week here for, for privacy concerns, so that way we aren't able to, to pinpoint the individual's base location or work location. Uh, we run through the typical steps of stop identification to identify the unique locations. There's a massive amount of preprocessing required here to make sure we're actually picking up unique locations as opposed to the jitter that's often observed in GPS points. We look at a number of destinations of interest, so retail and leisure. Uh, this is a large mall in the Lamington area of Nairobi that attracts multiple devices and users to that location, and then over the course of a week, we observed that there are about 200 unique visitors from this part of the data sample to that location and, and those retail facilities of interest. So what this helps us understand is not just where the people coming from across the urban fabric of Nairobi, but in their leisure time or their work time, where they spend that time and money. So I'll switch over to um. 3 fingers. Oops. Thank you. So, for today, uh, we put together a, a demo where we can look at that data interactively. So imagine that you're a government entity and trying to do road infrastructure planning or uh urban or if you're a telecom company trying to understand where existing signatures come from, but you know that there's a population that hasn't received uh self service yet as you're thinking about expanding. Your towers or a repeater coverage, this type of analysis can be useful. And so once the safeguards have been built in, we're able to then showcase some of the individual traces as well, to be able to pick up information like where the visits are and then be able to aggregate to these. So, uh, one of the advantages of planning for this architecture at scale is that we're able to deliver the insights directly to a variety of users as opposed to, uh, expecting that every organization that needs this information has the capability to, to crunch this information on on their own or implement the research methods in the way that we as researchers would do. So, uh, I'm probably out of time, so I'll stop there. Um, uh, one of the main sort of policy implications that we haven't touched on today that I'll speak to is the idea of procurement. For example, during COVID, we often receive calls from the bank on can we use this type of information to continue to support bank operations, uh, and also from ministries who were focused on other areas of public need like access to healthcare. To be or for example, to, to help deliver uh COVID vaccines in relation to where they were observing activity of people, uh, and so on in triangulation with, uh, population density, spending patterns and asset wealth. So there are a number of applications there, um, that, uh, aren't sort of causal inference oriented but, uh, more designed for immediate services. Thank you. Thank you, Vivek. Um, our next speaker is Barbara Kramer from IPRI. Yeah, thank you so much for having me. Um, also, um, it's just been really inspiring to see so many great talks today, um, and I think this is a good time for me to put in a plug for the journal where I'm, uh, editor of the Social Science section. Which is uh development engineering. and development engineering, um, publishes work um on rigorous research, on innovations to address global poverty problems or global development problems. Um, and then that includes a lot of the mobile data work that has been presented today. Both mobile data in terms of strengthening policies and programs, but also using mobile data to track impacts um of different interventions or to track um outcomes and. And so in that light also, it's good to mention that we currently have a special issue out on tracking progress towards the sustainable development goals um and if anybody is interested in submitting a paper um do talk to me or to Sam Fishman, um, who is also part of the editorial team and and keeps us as editors uh on track. Um, so having said that, let me now turn my eyes on the ground and talk about, uh, crop monitoring. Uh, through smartphone pictures of plots. Um, this is, uh, a project, um, that is joint work with Koon Hufkins, who is with Ghent University. Um, Lillian Weissaka and Benson Juguna, who are with Acre Africa, um, the implementing partner for this work, um, Acre Africa is an insurance broker and does a lot of work around, uh, smartphone-based data collection. Um, and then we're working with, um, Dwara e-registry, which is, um, an Ag FinTech startup in India, um, and the Kenyan Agricultural Livestock Research Organization. Um, so why crop monitoring? Why are we talking about, you know, getting our eyes on the ground on farms? Um, smallholder farmers, they depend on, uh, agriculture for their subsistence, and in developing countries, a lot of agriculture is rain fed and thereby is exposed, uh, tremendously to weather risks. Um, and, and this includes droughts, floods, also, you could think of pests and diseases. Um, and, and monitoring this, it's just really, really important to help improve early warning systems, um, to, to provide agricultural advisory services that would allow farmers to take preventive measures to minimize damage from these extreme weather events, um, but also financial products such as credit or crop insurance to help improve adaptation, uh, to and resilience to climate change. Um, now, advances in remote sensing have been enormous in the last few years to really improve monitoring of crop phonology and, and productivity, but there's still a number of challenges that are remaining, including some technical aspects. For instance, cloud cover in a rainy season, um, is is is limiting the accuracy of some of the satellite-based algorithms to predict productivity. Then there's, um, resolution issues. For small plots, there's intercropping in the sense that in a lot of smallholder farming settings and as a measure to adapt to climate change, we see that farmers are growing, for instance, beans and maize in one plot and how to make sense of that when you're using satellite remote sensing, how to estimate productivity in those cases. Then there's issues that are more social. People don't always trust what's being read from satellite images. It's not very tangible and so understanding becomes an issue as well. And finally, on the empirical side, there's a lack of ground truth data and a lot of models that have been developed, um and a lot of validation studies that have been done are focusing on major cereals such as maize, wheat or uh rice, but not on other crops that are very important for dietary diversity um and for vulnerable populations and more indigenous crops. So, with that in mind, um, we started a project in 2016, um, that is using smartphones to, we call it smartphone-based near surface remote sensing. Um, that is using smartphone pictures to monitor crops, and it works as follows. A farmer can download an application on his or her phone, uh, register a site using an initial picture, and then over the course of the season, take repeat pictures of that same site, and there's visual aids in the smartphone application that help tracking this. There's also GPS coordinates and timestamps being added, so that when the photos are uploaded to the server, um this information is all available. And then in the application that we're focusing on crop insurance, there's actually experts that are reviewing the images and indicating whether this farmer has suffered damage and should receive an insurance payout. And we did a formative evaluation of this in India whereby we found that indeed it is possible to track uh or to to do crop monitoring through this approach, especially to identify crop growth stages, um, which is useful in the case of advisories and to um. Uh, to, um, uh, identify severe damage which is useful in the case of insurance, and then the question became can we do this in other settings and so we started working on this in, in Kenya, um, and can we further automate image processing. And so in that light, um we did a project at a very large scale collecting images in Kenya um through in Kenya where smartphone ownership is a lot a lot lower than in the setting in India where we were working, um, we worked through champion Farmers, um, which are local opinion. Shapers who were equipped by Acre Africa with a smartphone and trained on how to send in these images and they registered a network of more than 100 farmers per champion and out of those we had 20 farmers for whom they took regular photos um over the course of the season for 3 seasons in a row. And all these images were labeled, um, and in terms of crop growth stage, type and extent of damage. So, um, that resulted in a data set of more than 13,000 maize images, uh, for the three seasons combined. Those have all been labeled, um, and out of those, we have more than 11,000 images. That are useful that are out there actually on the web um the images were also used to train convolutional neural networks um because so using computer vision to detect the labels that we had um uh created or that the agronomists had been assigning to the images um and and other seasons are being added over time. So, the reason that I was presenting this is that I think this is a great data set for those who are interested in agriculture AI and who have remote sensing um applications who are looking for Ground truth data sets. Um, this data is out there on the web. Um, I have a web a website indicated here. Um, and so also if you have further questions about the data set, uh, do feel free to get in touch, um, because we're happy also to share, for instance, a smartphone application if people want to collect this type of data in other locations. So, that's it. Thank you so much. Thank you so much Berber. It seems a recurring theme is that taking pictures of stuff with your cell phone is really great, whether it be bacteria or worms. Sorry, I can't, I don't remember exactly which. Sorry, Maria, um, OK, so our, our next, uh, presenter is Sarraj Nair. Uh, he's been mentioned a couple times in the conference and now here is the man himself, not to put you on the spot. Thank you. Um, Oh, awesome, thanks. Um, all right. Good evening, everyone. I guess we're almost done. Wine, woo-hoo. Um, yeah, I'm excited to talk to you today about, um, ongoing work with Josh Blumenstock from UC Berkeley. Um, we've been looking at the impact of cash transfers on mobile money usage and then more broadly digital financial inclusion. Um, in terms of motivation, I guess we've heard several times today that in the last couple of years, there's been a massive digitization of social protection. Um, I think this is the last accurate number. I have over 150 programs delivered aid via digital channels during the COVID-19 pandemic, um, and a large number of these programs relied on mobile money to deliver this aid, especially in, um, sub-Saharan Africa. Our paper focuses on this idea that delivering aid via mobile money or mobile money transfers could serve as an on-ramp to broader financial inclusion by providing access to very basic financial services for low-income individuals who are often excluded from the traditional banking system. So, this paper very specifically simply looks at this question of whether short-term exposure to these digital cash transfers can drive the longer-term usage of the broader mobile money ecosystem. I guess it complements this very quickly growing set of papers that generally look at the impacts of mobile money transfers. Most of those papers focus on welfare outcomes such as consumption or women's economic empowerment or intra-household bargaining or on efficiency gains for firms and employers. Much of this literature finds that short term exposure to cash transfers doesn't really change financial behavior in the long term. Um, at least in the past, pre-pandemic. We think this is going to change after the evidence from the COVID-19 studies come out. Um, the context we're focusing on is that of the OBC program in Togo. Um, Emily spoke at this, uh, spoke about this in the morning at length, so I'm not gonna spend too much time on this. I just wanna mention that we focus on a phase of the program. Um, that, that had a randomized rollout. So a random set of individuals were assigned to a treatment group who began receiving cash transfers in November 2020 for a five-month period, following which the control group received transfers. So we leveraged this randomization to study these impacts on mobile money usage that I'm going to talk about. Um. Yeah, that's the only thing I wanted to mention here. Um, I won't spend any more time. We, we use data from uh one of the largest telecom operators in Togo. Uh, this is a rich data set that gives us information on when individuals sort of withdraw and deposit money into their mobile money accounts, but it also gives us rich insight into their saving behavior, into the network of financial transfers, and how they're using the broader mobile money ecosystem to make digital payments and purchase airtime and to pay for things like cable and water and electricity and things like that. Um, yeah. So before I jump into the main results, I think my, the thing I really like about the analysis so far has been that simple descriptive stuff provides us a lot of insight. So, I'd like you to focus on this panel on the top where I'm simply plotting out the fraction of individuals making at least one mobile money transaction in a given month. Remember that the transfers we're interested in began in November 2020. So, prior to that, basically, roughly 20% of our sample was making a mobile money transaction. The moment people start receiving these transfers, you see this huge spike in activity in their mobile money accounts which um kind of sustains for the five months during which they receive transfers and then there's a decline of sorts. The interesting thing here, um, and I think this is what differentiates these results from, I guess, stuff that's out there in the past is Panel B where we're able to show that A lot of this increase in mobile money usage is actually driven by an increase in the use of wider ecosystem services such as bill payments and transfers and other things, so people are not simply using their accounts to withdraw cash withdraw cash and then spend it in cash locally. There's actual ecosystem-wide transactions happening in this setting at least. Right? And so this sets up sort of my one results side. I promise not to show tables, so it's all text. Um, but yeah, so basically, what happens when individuals receive transfers in a month, they receive transfer, we see this like 62% increase in the usage of their mobile money accounts relative to baseline levels, that's a 300% increase. Um, the nice thing, the encouraging thing rather is that these increases are ecosystem-wide, so there's people actually making digital transactions during this period. Um, what's, what we are trying to look at and get a better sense of is what happens in the longer term. So after the cash transfers conclude, do these gains persist or does everyone go back to using cash? And I think at least our initial results suggest that there are modest gains in mobile money activity, mobile money account usage in the longer term. Our data lasts for about 4 months after the cash transfer program ends. What I'm showing you here are results from an event study analysis where we're comparing individuals who received a cash transfer to individuals who never received one. So, relative to baseline levels of mobile money usage, these results show the same thing. There's this large initial increase in the months during which they receive a cash transfer, and then there's a drop-off of sorts in mobile money account activity, but Even 4 months after the cash transfers conclude, individuals who receive transfers remain much more likely to use mobile money than individuals who don't receive these transfers. So, that's the broad pattern we see. Um, we think there are some specific, um, factors in Togo that contribute to these increases. Um, Togo, they subsidized the transaction costs to withdraw money, at least initially. Uh, we think this helped. There's a dense agent network and the Togo government was also running these financial literacy programs even before the pandemic, all of which could have helped familiarize people with some of these aspects and um. Lead to the results that we're seeing. What are the implications of this? We think the NoviC program, based on rough back of the envelope calculations, added at least 80,000 customers to the ecosystem in the long term. Um, and again, using like reference numbers available from exercises that have been done in West Africa and Kenya, we think this means up to 80 million um Togolese francs in additional revenue and potentially up to 160 million in terms of true economic value. Uh, added to the economy. But these are back of the envelope calculations, we're trying to be better about this, um, but yeah, that's my results. Um, thank you. Feedback is welcome. This is the first time we're presenting these results, so we would love to hear your thoughts. Thank you. Thank you so much, Sarraj. Um, next up we have, uh, Winnie Mugogu from Busara. Uh, we're really excited to have her and thank her for, thank her for coming all the way to Kenya to pre from Kenya to, uh, to present, uh, these, uh, results, and I'm hoping that while I was talking, Napur would come to change the slides, but that has not happened. All right. Oh, there we go. Cool. Um, did it all by myself. All right. All right, Winnie, the floor is yours. OK. Hello, everybody. Um, today I'll be presenting on um moving from in-person to mobile data collection in Kenya. I should have said at Busara, but the at Busara is silent. Um, I'm a research specialist at Busara Center for Behavioral Economics, which is in Nairobi, Kenya. So I'll just give a background on Busara. Um, methods, research agenda that we have going on and some studies that use mobile data collection. So I'll highlight a lot on the lessons learned from these studies more than the specific results. If anybody wants, is interested in any of the studies and the specific results, they can consult me on that. So, background. Um, Basically, BUSARA Center um is a research institution and a consultant um they do consulting work with researchers and organizations to advance and apply behavioral science in pursuit of poverty alleviation. That's the mission of BUSARA. So basically, we have a nice behavioral lab in Nairobi, Kenya. Possibly the only one in Africa, but I could be wrong, so anyone can correct me on that one. but we um conduct some behavioral studies in Nairobi, Kenya. In March 2020, I moved from my home country, which is Malawi to Kenya. And I arrived in Kenya and a few days later, there was the first COVID case. A few days later, they said we're not going to work next week. So I never actually got to see this lab and I never got to experience um all the wonderful experiments that people carry out. But as a research institution, of course, we had to somehow carry out the research. We needed Data. Um, as such, yeah, the pandemic, it came in the way of um data, in-person data collection as most of the studies today have been motivated by, you know, the pandemic and so we had to think of how do we continue to collect data, otherwise, we can't carry out our operations. And so we decided, well, remote or mobile data collection and this is in a low-income country setting um We had the subject pool of respondents in Kibera and Kwangware, which are um urban slums in Nairobi, Kenya. If you see a lot on slums in the news, it's probably in Kibera, very um the largest urban slum in Africa, actually. Um, and so as we were thinking around how do we continue um data collection, that's when we came up with this methods research agenda. So I had the methods research agenda at BASARA and basically, we are, we have a few questions that we're trying to answer. I've just selected two of the most prominent ones um that we're trying to answer with this methods research agenda. It's how we can contextualize, measure and improve access. Response, attention, comprehension of surveys and experiments for populations in the global South. And this is going to be specifically be applied to remote data collection because that's what happened during the COVID pandemic. We couldn't do in-person data collection. So, we're looking at, you know, still improving access, still getting good quality data, still getting um you know, good attention even though the, the research is being carried out remotely. And then how do you, how do the answers to these questions vary across gender, racial, national, and economic groups? I think a lot of the discussion today as well has been, you know, around um do we see differences in gender when we're using mobile data collection. These are some of the questions that we're going to be answering and some of the studies that I'm going to share. Um, begin to answer those questions. It's going to be carried out over the next 3 years. If anybody's interested in collaborating to answer these questions, um feel free to do so. So, One of the ways we innovated was through what we called Kite. Kite is um which sought for Knowledge and Insights tool for experiments. So basically, we wanted to continue doing this behavioral econ um experiments but on, on a mobile app. Um, so It was created in-house in Bossara and one of the studies that we um we did on the app was a financial safety nets and attention study. By that, it was how are people coping over the COVID pandemic financially. I think these are important questions to ask at a time where we couldn't visit the people in uh like we couldn't go in person but we wanted to know how they're doing financially. Um, and then we added in this part of attention. Now, attention is because when you're thinking of remote data collection, you don't have the enumerator or the FO to the field officers to keep people's attention going. So how are we so sure that the um with remote data, people aren't just clicking away at answers and then submitting? I'm sure we've done that sometimes when you get an email asking you to complete a survey and just kinda click, click, click submit. Um, and so we try to Answer questions like the sources of income people used before and after the pandemic, but also the attention part would be would ask questions like, were you born before 1940. The inclusion was people who were a specific age and there was no way they should be born before 1940. That's how we would measure something like attention. So, we ran this with um a target sample of 500 people but you only reached 375. Um getting people to download the app was quite a challenge um and so we're actually pivoting how we're going to use Kite so that we include a lot of people um given how difficult it was to get people to download it. Um, this is a study of, uh, chlorine, uh, the study's actually called Can Simple Psychological Interventions Increase Investment in Preventative Health. I've included this study but it's by Kate Orkin and Annette John. Kate Orkins at Oxford. Um, collaborated with Bosara. When I'd arrived, we were supposed to do a follow-up survey in person. We had to pivot to, um, a phone survey. We managed to reach 3,194 women in rural Kenya, in Bungoma and Kakamega. And to reduce attrition, we went and tracked respondents on the ground, but they still did the surveys over the phone and for those who didn't have a phone, the people who did the tracking, they borrowed the respondent the phone so that they can still take part in the survey. So that's one of the ways in which you try to include people who would otherwise have not been in the survey. Um, I'll skip this, uh, study on we use SMSs because there's the one study which we found very similar results that was just presented, um, and cause of time. Um. And then we've also tried um to do call tricks and compare that to phone surveys. I'd actually initially wanted to present on this study on using the LT experiment, which is an indirect method of measuring intimate partner violence, um comparing it over result um findings on court tricks and on a phone. It has taken quite some time to start data collection because of IRBs and you know, the sensitive issue of IPV but we finally got an approval. We're starting data collection this week if anybody's interested to find out. Um, just the idea behind this um study with the list experiment is that um with intimate partner violence, with sensitive topic. topics, people aren't so open to share about their experiences and so the idea with this methods research agenda is that we want to make sure that we are collecting good quality data. And so today, the idea is how do these things, how do these things affect policy? What, uh what we're trying to get from these studies is basically that Policy must be guided by evidence and can we trust the evidence? We can only trust the evidence if they come from good methods and that's why we looked at things like this. So um that's why I had to share. Like I said, there's so much I could have shared specifics of the study, but yeah, I just didn't have time for it. Um thank you. That's a great quote. Uh, you can only trust the evidence if it comes from good methods. Um, so our last speaker of the day, uh, is gonna be, uh, Zach White from GSMA, which, as we've seen in, in, in several of the presentations of prior speakers, you know, really makes possible a lot of, uh, a lot of the research. So, um. Yes. All right, great. Thank you. This is it. That's it. Fuck. Uh, so I really am standing in the way of the last thing. But I'm gonna do 3 things today which is just introduce the GSMA and then move on to 3 frameworks that we use to look at partnerships, uh, and particularly around mobile big data. And then I'm gonna talk through some of the cases where the GSMA played a very active role in facilitating those in the context of the COVID-19 response. And you can count them down as 321, should you choose to. Uh, so the GSMA and talking to people it seems about 50/50, uh, those who know us, those who don't. It's the industry association for the mobile industry, so our members are about 800 of all of the major mobile network operators and associated ecosystem players. Uh, we convene the industry, there's industry services and events, but then also within the mobile, uh, the GSMA is the Mobile for Development Foundation that sits at the interface of development and the industry. And I am one of, I work with the Digital Utilities Program which is one of 9 kind of thematic programs working across, you know, agriculture, mobile money. Uh, uh, and many others that my brain can't recall recall right now. Uh, and within the utilities program we focus on energy, water, transport, sanitation, and waste management, those services. And what we do in, this is similar across all of the programs and portfolios is most importantly, one is we give innovation grants uh to predominantly startups and SMEs. To date we've given about 100 grants totaling nearly 20 million to about 100 organizations. Facilitating with those about 80 partnerships with MNO's and I think the kind of headline numbers are within 2 years, those organizations went on to raise over half a billion, something like 530 million and impact 36 million people. Alongside that we do research and insights and uh convening uh. For the industry. So that's the first bit down. Number 2 is the frameworks we use to think about uh partnerships. And this is the first one which is also, it's a process but it's also the different stakeholders and it's also the technical aspects that come in. So starting with the MNO data, it distinguishing between the different types of data that come through and being clear about which you need and then the process that goes into making those useful within the warehouse. Secondly, where it comes in the third party data and how we've talked about combining those and the specific analytics you need to get those useful uh packages out for the decision makers. And of course these can be different people across different stages. Uh, so the journey map for partnership describes those elements that are particularly important, and I think a lot of people will be familiar here with the partnership establishment of how you get to that first stage of getting the data and then similarly, the issues with the data management and the integration. But I think what maybe the research community doesn't think so much about is then the sustainability of these, the financial sustainability. And then also ultimately the impact and usage and how it all comes and the fact is you can move between these different elements or look at them in a different way as different starting points. Um, I'll skip part this this very quickly, but we use the haves and needs framework quite a lot in partnership establishment to really think about where the value's coming from on both sides and to be clear about that from the start. That's frameworks down, 23, COVID response is uh led by our AI for Impact initiative which sits outside of Mobile for Development. But this is a task force of 21 mobile operators covering 2 billion people um that provide that has an advisory panel and and works to develop specific initiatives around mobile big data and AI, um. We've seen this already with fair to slide, and as mentioned, I mean the big spaces of white where not a lot of work went on. Uh. So the GSMA worked with support from the UK Foreign and Commonwealth and Development Office FCDO uh in about 14 countries, providing there some advice, some technical advice, some consulting on ethics and and business practice, but in 4 countries, Benin, Burkina Faso, DRC and Rwanda, we worked more intensively to facilitate some public-private partnerships and, and they're what I'll talk about in a minute. Um, so I think this is also very familiar to, to what we've been talking about today, but going back to that first slide and thinking about. You know, during the process what could have been uh informing these partnerships, where different data and particularly from different agencies needed to come in to produce different sorts of analytics for different sorts of decisions. And this was really uh some of the thinking that was done at the very start of the process to determine what happened in each of those four countries. Uh, and it's slightly out of place but that's the output for DRC in terms of the dashboard on kind of mobility. Um, as well as, um, some other analytics. Uh, so this is revisiting part of that but just to kind of package, uh, exactly the pro the process that the AI for Impact team went through in those four countries. Um, I'm gonna skip past it to, to some of what came out. Oh, that hasn't scaled very well. Um, but, uh, I think, you know. It was, there's 6 things that that we point to that we were proud of within this work. I think the first is that you know it was a successful use case development for mobile big data products and to develop and deliver those in 4 countries producing data for decisions in a very short time frame uh was an enormous achievement. Uh, it increased awareness among some of those key stakeholders. Many of them were learning that interacting with these products or concepts for the first time. It promoted a state of privacy and governance, so kickstarting a lot of discussions that that you know perhaps hadn't been considered before, especially through the the government agencies that were worked. Uh, there was a lot of kind of enhancing of technical skills of those key, key actors, um, hopefully laying some of the foundations for, for the ecosystem growth, uh, and, and then also our role in, in efforts to increase knowledge sharing around this. Um This maybe you can revisit the slides in your own time cos I'm I'm running out of time. er but this this draws on some of the other work that we've done around innovative data for urban planning and points to you know the checklist of things that that really should be considered in in partnership formation and also could present stumbling blocks at different times. Uh, the penultimate slide is just some of the resources that you may want to visit and it's it's kind of more in there for. Uh, revisiting at a later date, but to, to flag some of the key resources on ethics and and principles for using, uh, big data AI. Importantly, and something that hasn't been talked about a lot apart from in this lightning talk session is, is the business models that go with that. Uh, especially if you want to, to have sustained use and you know where third parties come in to apply these analytics to to produce useful products and services. Um, and then there's also some of the utility specific resources there, and I think with that it ticks off the final point for the day and thank you very much. Um, all right, well, thank you so much. Um, I have a couple of closing remarks. Um, so first of all, I wanted to thank you all for being here and it's so, um, exciting and it's so valuable to be here in person and it has been like truly a different experience for me because I've only been to maybe two conferences in person. And the amount of that I've gotten out of today is just astronomically different than what I would have gotten out of it, you know, virtually or in another situation. So I hope that that's the case for you and um, yeah, thank you so much for, for, for coming here to the World Bank. Um, another benefit of in person is that we all get to like chat, uh, and not in a breakout group. So, um, I hope that any questions that you have for some of the speakers for the lightning talks, for example, you're able to ask them, uh, during the reception. Um, I just wanna make a note for Berber. There was a specific question for you. Oh, addressed, perfect. OK, great, um. Next, I really want to thank, uh, some of the co-organizers of this event. So some of them are standing right here, um, they've been wanna come out and and give a little wave or something, um, so these are, uh, the people who helped organize this event and have kept it running, and it is truly like, uh, a large effort on the back end. So let's give a round of applause for.
showAllTranscripts
no
duration
PT50M
scene7File
worldbank/Measuring Dev. 2022 Lightning Talks
scene7Domain
https://worldbank.scene7.com/
scene7FileAvs
worldbank/Measuring Dev. 2022 Lightning Talks-AVS
title
Measuring Dev 2022 Lightning Talks
description
Measuring Dev 2022 Lightning Talks
showTimestampAndTranscript
yes
col-xs-12
col-sm-12
col-md-3
col-lg-3
col-xs-12
col-sm-12
col-md-7
col-lg-7
  • add-style
  • lp-body-content
Measuring Development 2022: The Role of Mobile Data in Global Development Research — Lightning Talks Recording
lp-heading-top-medium
lp-heading-bottom-medium
col-xs-12
col-sm-12
col-md-2
col-lg-2