Our next session is session 2B mobile data, uh, and COVID-19, so that will be chaired by Norbert Shady, who is the chief economist for Human Development in the World Bank Group. Um, we have two virtual speakers who unfortunately were not able to make it, uh, to the conference. One is in Ciudad de Mexico. The other is in Italy, so it was a little bit difficult. Um, so we will have, uh, those speakers, uh, present via the screens. Please come up. The MI. It's assigned seating over here. OK. I'm a bad pupil, evidently. I didn't learn to do as I was told. Um, would you like me to start? Well, uh, thanks to the organizers, thanks in particular to Ariana for asking me to chair this session. It's, it's wonderful to be here. Um. Maybe by way of background I wanna say a couple of words uh we all know that the pandemic has had uh disastrous consequences for um uh human development and well-being in in in many different dimensions um including in health as we just heard before but also in terms of schooling in terms of uh labor market outcomes, um, and, and it's really something that I'm leading a study from uh from the chief economist office which is looking at, um, uh, what these impacts are and what to do about them. And in that context I think uh these discussions about to what extent can technology help us address some of these problems, circumvent some of the constraints that we've normally seen is extremely, extremely timely and I think we'll all learn a great deal, uh, from this session. So I'm really pleased to be, to be, uh, um, uh, to be chairing this session. So, uh, let's start then. I just got a, uh, note that. Uh, Lorenzo might not be able to join. Uh, if so, go to Victor first. So, uh, this is all a mystery to me, but, um, so, uh, let, let, let's start then, uh, I guess maybe with Victor. Is that right? Yes, we start with Victor. Great. So, uh, so Victor Orozco, uh, he's a senior economist, uh, here at Dime. Um, he leads the research program in mass media and entertainment education. And he's going to be telling us a little bit about using social media campaigns to promote health seeking behaviors of households and just to make sure that we all have time to listen to these presentations, I'm going to, I'm going to mute myself now and pass the word to Victor. Thank you. Thank you for the invitation. I'll be talking about a work we're doing with um a series of partners including Facebook, the World Bank Behavioral Unit, MIT, uh NORAD, um, and We are, this work, it's really a portfolio of work I will be describing today, aims to close the digital divide when we are targeting development campaigns to social media users, especially poor and illiterate, a key target population. And this is done through 3 types of work we are doing. First, through a series of surveys and experiments to understand the constraints in terms of attitudes and behaviors people are facing for adopting new behaviors and to inform the content of new campaigns. This is in the context of COVID campaigns. The second type of work we're doing is experimental studies to to test actual social media campaigns as well as the distribution strategies. Often we focus too much on the content, but then we forget on the distribution. And lastly, ongoing work in developing new platforms to to to to proxy learning and poverty levels of households. At the end of the day, a lot of the geographical targeting can take you up to a certain degree, right, closer to your target population. As well as testing social and behavior change communication when combined with development apps. You, you can have a very punchy, inspiring behavior change campaign, but if you want to teach a kid to learn, you better also provide learning tools uh to the household. That's how I'm going to be closing. So as motivation, it's clear that social behavior change communications can reach billions of individuals. Over half of adults own a smartphone today. There are 1.6 billion Facebook daily users right now for social behavior change communications to work, they need to be persuasive, behaviorally informed, including the Formats of edutainment, narratives basically for reshaping attitudes and promoting behavior change. Information only campaigns are good for raising awareness, but usually cannot change deep seated attitudes and behaviors. The second point is reaching the target population. Many campaigns have a very generic naive targeting strategy, often restricted with geographic. Targeting at best, so you know we may not be reaching our vulnerable populations, for example, people that live in malaria risk settings. But of course a lot of the optimization routines of online platforms, as well as users' behaviors are really driving the targeting, the delivery of these messages, so. We need to take a few extra steps for a smarter targeting of these social media campaigns. And uh again, as I mentioned earlier, complementing behavior change communications with uh apps for long-term impacts, for, for learning, for example. A recent literature shows that a lot of social media campaigns with development objectives are based on engagement measures like like spans, and websites. But of course for development researchers and policymakers, we care about impacts on more medium-term outcomes such as knowledge, attitudes and behaviors, dying with. Researchers developed virtual lab, which is basically an open source chatbot that delivers a chatbot service like the one you're seeing on the left, but also allows you to deliver interventions and measure objectively online information seeking behaviors such as if an individual clicked on a website. Um, or, uh, adding in the Facebook profile picture a banner supporting a social cause as a public display measurement of support. Um, Virtual app currently runs on Facebook platforms, and, and the goal is to be expanding it to, to other platforms in, in the short and medium term. Um, in terms of service and experiments, uh, in, in January 20 of last year, uh, the, the World Bank launched a program to support and understand vaccine hesitancy. This work is being led by ABT, um. By the end of this year they've covered over 17, they've covered 17 countries and with over 140,000 respondents in all regions of the World Bank, and these surveys to understand constraints but also testing some framing of such potential campaigns are informing. Um, uh, the contents of actual campaigns, uh, here's a selection of some of the lessons that have been learned, uh, summarized very nicely in a World Bank, uh, uh, blog, which is at the bottom here. Um, so for example, safety concerns are, are, are driving most hesitancy. People trust certain messengers more than others, right, such as healthcare workers and friends, um. And that health workers, vaccine hesitancy remains high among health workers, similar to the levels of the general population. Um, now, uh, this work also entailed, uh, running a series of multi-arm multi-treatment arm variations of campaigns where we could see that multiple messages were tested and adapted with message improving intent to get vaccinated by up to 80% with respect to a control group. So, it also opens the door to be testing multiple variations before any at-scale intervention. The second portfolio of work is about a randomized, a couple of randomized controlled trials we're doing in India of a real world campaign aimed at preventing malaria. This intervention conducted by the MalariaM reaches 133 million people in 22 states. The campaign targets high burden states and it's stratified by age, gender, and metro rural locations. However, as, as many of you know, demographics that are easy to target may not be associated with the outcome of interest. Through formative research, we find that living in a non-concrete house, a dwelling versus a concrete dwelling is a major determinant for malaria infection. We use Facebook lookalike audience tools to to to proxy successfully households who live in non-concrete dwellings, and then we stratify our study for non-concrete as well as concrete households. It's a randomized controlled trial that takes place in 80 districts in the states of Uttar Pradesh, Janghahand, and Jatishgarh and, The NGO basically conducted its normal campaign, right? Respected the the the the the the districts where they were supposed to conduct the treatment and didn't go to the control uh districts for this cluster randomized control trials, and then we separately recruited our our studies uh participant. Pants are surveys. Something very interesting of this research is that we combine a chatbot, chatbot survey with health facility given that we uh um randomize at at the health district level, allowing us to triangle it with administrative records. Here you can see a map of our treatment and control groups health districts. We find that this ad campaign was very effective in increasing the use of bed nets in concrete households, but did not work for people living in non-concrete households where malaria risk is higher. Analysis of our health facility data confirms this. Well, urban monthly incidents decreased by 1% point on average, or a quarter of a standard deviation 1 to 4 months after the campaign, rural incidents was not affected, right? So the question is, was it punted or targeted? Did it only work for urban areas because on the was appealing to the urban areas or simply that it reached better urban households where access to social media is greater. So using the remarketing tools of the ad platform in a second trial and individual level RCT, we experimentally varied exposure to the same advertisements for 50% of individuals in our second round of data collection, we collected, I believe, 10 rounds of follow-up surveys. Where individuals are now exposed to an average of 6.5 ads during two weeks, and here we find that This time, the intervention did work for both urban and rural, suggesting that the ad content worked, but additional efforts are needed to reach target population and, and well, this study is helping us, helped us scale up the partnership we have with Facebook and Embed where we're working in multiple countries. Lastly, I'm going to mention some of the platforms we continue developing. for targeting poor households, for proxing poor poverty levels of a household, as well as proxying learning measures. Now this benefits from work we did in northern Nigeria where through a cluster randomized controlled trial targeting offline populations, this 5 day intervention combined entertainment screenings with the provision of smartphones that Included gamified apps and a digital library, and this 5 day intervention was super cost effective. It improved learning outcomes of the targeted child and important spillovers on other household members in terms of learning, a reduction in teenage pregnancies, and delays in early. Uh, to the labor market. Now, the question is, can this approach be effective for online populations, right? Basically, social media users, right? So we're currently doing a multi-country tech trial in Sub-Saharan Africa, where 87% of children are learning poor, um. And of course, engaging social behavior change communication may not do the trick, right, for teaching children how to learn. So the idea is to do this trial in Nigeria, Senegal, and Tanzania, but we are currently already running a series of pilots in Bangladesh. Where for example, we, we, we are uh piloting an open source gamified platform that assesses early reading and math skills, and where um it's, it's modules are highly correlated with the modules of the gold standards of EGRA and EGM tests for measuring these outcomes. Uh, also in Bangladesh, we are testing a pilot we plan to use for this uh multi-country tech trial in sub-Saharan Africa, where we, um, in addition to geographical targeting, right, targeting poor neighborhoods, we, we want to do a better job in targeting poor households. So to identify socioeconomic status of online population. We developed a short voice-based gamified survey for illiterate populations to ask anyone in the household about poverty-related indicators. So through machine learning approaches and using the Bangladesh Household Income and expenditure survey, we are selecting good predictors for poor basically in Bangladesh and here you could see how this works, right? So so so the person, including a child, can see is asked how does your family cook or what does your house look like and 7 or 8 other similar questions to determine poverty. Um, I hope I didn't go over my time, uh, too long. Uh, main takeaways, uh, the public sector needs to invest in high quality behaviorally informed content and distribution campaigns. Um, budgets for distribution campaigns tend to be small and restricted to the project cycle. That, that's, I think, a major issue. The public sector needs to invest in public goods, including off the shelf tools for, for targeting those most at risk and maximizing the social impact of ad campaigns. Uh, a lot of the off the shelf tools currently are for private sector marketing campaigns, right? Did the person click, did the person buy, but we want to measure impacts on medium term outcomes such as knowledge and attitudes and behaviors, and for that we will need to be developing more of these off the shelf tools. Lastly, Um, we need to invest more in scalable and cost-effective innovations, including social media campaigns, development apps such as uh these air tech learning apps I was briefly describing, uh, we tested in northern Nigeria, and, and, and measurement tools such as learning poverty proxy. Uh, the big advantage of this approach is, is that they are sector agnostic, right? It it's a lot of programming and they could be used in different sectors in different regions with relatively minor uh tweets, um, and across different development sectors. Um, that's all from my side. Thank you. Uh, thank you. That was really fascinating. I actually just want a couple of sentences. One is, uh, on the malaria intervention. I thought it was fascinating at first it worked in urban versus but not in rural areas, and that was somewhat, um, worrying given that the incidence of malaria was so much higher in rural than in urban areas. But I really like the fact that they had an iterative approach than to figure out. Um, why it hadn't worked or how to make it work in, uh, in, in rural areas, so that was a little bit along the lines of, uh, you know, the, the economist, this plumber kind of work that Esther Duflo has, um, has, has pushed, um, so I thought that was great, and on the, on the learning outcomes, I think this is really, really important given the massive learning losses that we're seeing from the. Pandemic, uh, we know on average that children who missed 6 months of school, missed 6 months of learning, uh, overall they learned nothing while they were gone. We know that from a lot of impact evaluations now, so figuring out what kinds of interventions will help them bring back that learning is really, really important. Uh, we're next gonna go to Arman Reza. He is an assistant professor of economics at UC Davis, uh, and his research focuses on intersections of service delivery, of political economy and technology. We're really looking forward to, to this talk. Uh, thank you very much and over to you. That summer. My slides are not nearly as pretty unfortunately. I'm gonna be stuck with latex. Yes. All right. Uh, well, thanks so much for having me. Um, it's really awesome to be here and be in person. Uh, I'm gonna follow Victor's talk, uh, really closely in, in at least two ways. First, I'm gonna, you know, say, cite that, uh, research as a reason why the fact that we're only gonna go as far as engagement in this talk is OK because other people are looking at the relationship between engagement and outcomes. And second, I'm gonna pick on it a little bit for my motivation. So, The motivation for this study is one that we're very familiar with, which is that social media. is and can be a powerful tool to reduce the cost of information sharing. So imagine you're a large international NGO and you want to get across, uh, information to a bunch of folks across the globe about how to reduce malaria risk, right? You can take to social media to get this information out there and lead to social and behavioral change. And this can allow for the increased dissemination of what I would call useful or accurate or helpful or beneficial information. But it can also lead to the increase of harmful or inaccurate or misinformation. So imagine you put out a nice infographic and some influencer takes it, you know, draws a crude thing on it, says fake news and retweets it and gets more likes than the original post, uh, you know, this can happen and does happen, and you might worry about this type of spread of information on these networks. And you know, there's no time, at least since 2016 that this type of misinformation and that, you know, the both the positive and the negatives has been so apparent in on social media than with COVID-19. So there's a lot of prominent examples of social media platforms and, um, you know, public health agencies using. Uh, Facebook, Twitter, other networks to get out information about vaccines, about mask wearing, so on and so forth, and probably quite effectively, though I haven't seen any studies particularly on that, uh, and there's also a lot of examples of misinformation leading to probably harm on these networks. And you know, I don't need to motivate it in this context, but I will say, of course, that. This type of thinking about the costs and benefits of using of social media and the information that spreads on it is particularly relevant in developing countries where baseline levels of knowledge and education are low as we just saw also where there's fewer hospital beds. So if someone does something stupid because of what they saw on social media and ends up in the hospital, it could be much more harmful in a setting with. No beds, right, which of course is also true in the developed world, so this is something general but maybe particularly true in a lot of low income settings. And so we're in this case gonna take the role of both the folks trying to disseminate helpful information and the social media network itself. And we're gonna then be able to ask a couple of research questions related to this motivation, which is one, let's say you can actually fully control, so you are the social network operator, you're Facebook, or in, in our case, I'll, I'll tell you who we are in the next slide, but you can control all the information on your network, OK. So what if you could put out good information and at the same time remove the bad or the misinformation? How does this, you know, what are the trade-offs here? And the trade-off that we're gonna highlight is specifically in the act of trying to control and remove misinformation is going to cause people to get upset and maybe use your platform less, and then they're not gonna get exposed as much to the good information you're trying to get them. OK, that's the exact trade-off we're thinking of. And we're gonna do this through an RCT on our social media platform. We're gonna actually have two arms for the uh moderation control side. One is just removing all misinformation related to COVID. The second is gonna be something you've probably seen other social media platforms do, which is we're gonna sort of pin and add a rebuttal to misinformation. And so we wanna not just compare sort of moderating and removing versus not, but also ways of doing that moderation. What's our social, uh, media net, uh, platform? We are working in Pakistan on a voice-based social media platform called Bong, which is the Urdu word for a rooster call. This is just a generic social network, OK. We, uh, are working with the, you know, one of the co-authors is the person who created this and launched it, and it just operates like any other social network. Uh, we are using it to then do a bunch of campaigns to try to get across public health information, combine it with telemedicine, do a bunch of stuff, and then when COVID came up, we thought this would be a good opportunity to, uh, get some information out relevant to COVID. So think of it as Reddit over the phone. You call in, there's a high level menu. You can choose to, you know, listen to other people's posts either sorted by most popular or by newest, and you can make your own post. And these are all audio posts. And if you listen to other people's posts, you can then comment on them with an audio post, listen to comments. You can also like, share, comment, or sorry, like, share, just like report standard stuff. OK, so it's think of it as like one. Landing page of Reddit, but again, all over the phone, and it, uh, and just running, uh, kind of on and off in Pakistan over the last 10 years. Or 7 years. OK, so we, you know, we're running Bong, we're doing some other stuff on it. COVID-19 hits. So the first thing we say is, well, we should get some information to people because this is targeting poor rural folks across Pakistan who might not be listening to the news or checking the, the Pakistani NIH website. So let's get them some information on masking, social distancing, at the time, you know, washing your hands a lot. And so we just prominently posted a bunch of information that we, you know, carefully researched and recorded with voice actors and then tried to make as persuasive as possible, uh, and we made it like a prominent, you know, menu item number one. On this network, we just put it there, OK? That was the first thing we did. And we did this as quickly as could, as we could, so around April 2020. Then, you know, a few months later, we launched the experiment on top of that, so that just all users have access to this information and then we launched an experiment which we ran for about 2 months where we did this moderation. Randomization. And so, as I already said, we had a control arm where it was just business as usual. So this is community moderation. Everything goes live instantly. People will complain or report it, then maybe a human moderator will uh check and potentially take it down and practice. That never happened. OK. And then two moderation arms, all of these are both fully ex ante. So human moderators listened to everything before it went live. They classified it as unrelated to COVID-19, which is 99% of the stuff on the platform. This is just a general social network that just goes live. And then related to COVID-19, if it was neutral, it goes live. If it was useful and we, we sort of hand coded everything and went through multiple steps as whether it, it could be useful. So it's a, a user generated post, but one that has some beneficial information or whether it was misinformation, OK. And it was only in that last category of misinformation that then the moderation bound and it was either removed in one treatment arm forever or it was posted but with an official rebuttal from our team, kind of in a similar format to the, the, the post we put up earlier. And uh these are just 3 treatment arms, did an experiment. Everything else across the platform was the same, 99% of the content. It was all shared. OK. This wasn't like 3 separate arms. It was one big network. Uh, you know, we're not Facebook, we don't have the same number of users, but we have, uh, you know, 120,000 calls during the two month period of our treatment, uh, 13,000 posts, so on and so forth. So, you know, a fair number of active users and engagement and including engagement with our official posts. Uh, I'm gonna skip talking about our users and just jump into some results. Uh, I'll show a couple pictures, one table. Uh, so we're, and we're just gonna look at engagement. In this study, so engagement here is gonna be in terms of minutes of exposure to certain types of information. So first we're just gonna start with general overall minutes spent on the platform listening to actual content, so not in the menus and whatnot. And this is all in the post period. Uh, there's a, a halfway point where we ran out of money and so we had to make it no longer free and we, uh, made it, you get like an hour free a day and so that's sort of there was also some other stuff going on, but you see it, you know. In terms of overall usage, both on the extensive margin, number of users per day and the intensive margin, minutes per user per day, the control group has more usage than both treatment groups. That's the takeaway here. Then we can say, well, what about not just general exposure or usage, but let's talk about exposure to official, these official posts we put up. So our people listening to this information we think is gonna be helpful for them. And we see a similar pattern, though here it's mostly on the extensive margin, which makes sense because there's only like 7 minutes of content, so most people that once they start listening, they kind of finish it. Uh, and so people are using the platform less. Maybe they're also using it differently. We can't disentangle these, of course, post-treatment, uh, or, you know, it's hard to do so, but, uh, you know, that less usage seems to be translating to less exposure to this good COVID information. Uh, we can average, of course, you see it's stronger in the first month than the second month, so we're gonna average across the whole period and we see about a, I believe it's, you know, 26% on the control mean decrease from treatment pooled across the two treatments and listens to those official posts. We see a similar and, you know, stronger in terms of uh relative size, decrease in, in exposure to what we call these useful minutes. So again, this is user-generated content related to COVID-19 that we deem, so us, the researchers deem as helpful. So, you know, someone saying, I, I wear a mask and I, you know, I feel like I'm being safe. Some something like that, OK, and. Then the last column here is exposure to misinformation, and here, you know, mechanically, we see it goes to zero in the remove group, so it worked. We, we, we decided it was a misinformation, we took it down, you're not then exposed to it. Uh, we also see mechanically it's not going down in sunshine because we're not actually taking in that information down, so the pooled effect is sort of a zero or a slight negative, uh, but it was working. It, it worked. So we, you know, we're seeing this decrease in exposure to misinformation, but also this decrease in exposure and engagement with the, and also with engagement, we actually have measures of all the other measures of engagement and the pattern is similar, OK, so any sort of engagement you want to call it, we see a decrease. Uh, and so then we spend a lot of time then thinking about these trade-offs, OK, and this is where, you know, we're working on a, a framework to actually put some, you know, social welfare functions and all of that, uh, to the problem, but we also measured trust. We're, you know, really thinking about, well, it's not just the information and the different sources, but how much they trust, and if anything. People trust our information more than user generated, so that even goes and skews in the favor of the fact that basically the decrease in exposure to good information swamps the decrease in exposure to misinformation in our context from moderation. So sort of in a, in a equally weighted by by minutes sense, it was a negative to do the moderator. In this context and um you know I wanna, you know, caveat of course this is one particular context we have a lot of information and share it with our paper about sort of what our population looks like, what their usage pattern looks like, what their engagement looks like. I don't know how that compares to other social networks because they don't share this type of information as often, uh, but at least we're transparent, uh, and I didn't get to talk about mechanisms, but we have some, uh, sort of evidence that it's probably. Related to just the fact that moderation causes delays in your post going live that might cause people to stop using it. So it might be something that as we increase, improve machine learning and the ability to moderate quickly, which is a very hard problem, especially in audio and other languages, so it's not anytime soon, but maybe we can, you know, get to the best of both worlds. But I'll end on the policy implication here which is really just we want to identify this trade-off and tell people they should be thinking about it. So if you're going out and trying to really control information. In a, in a way that people are gonna notice or feel that can have the positive effects you're looking for but also some unintended consequences in terms of overall engagement, overall usage, and thus exposure to the natural sort of good content they would be exposed to through the course of just, you know, doom scrolling or whatnot. So with that, uh, sorry for going a minute over, appreciate it, and thank you. Uh, thank you very much. That was obviously, uh, fascinating and, uh, and very relevant, uh, in today's environment, given, for example, Elon Musk's acquisition of Twitter. So, uh, maybe you should send them uh your paper and, uh, and you can reflect on that, uh, but maybe, maybe he has other things to do with this time. One never knows. So, um, let's move to the last, uh, presentation. This is by, uh, Lorenzo Lucchini, um, he's a research fellow at the Department of Social and Political Sciences at Universita Bocconi in Italy. Um, so without further ado, uh, uh, Lorenzo, um, the floor is yours. Can you hear me? Yes. OK. Uh, so, thank you for inviting me. Today, I'm gonna present you a multi-country analysis of individual mobility behavior during the first year of the COVID-19 pandemic. Um, in, in general, every, every, we know that every crisis comes with a cost and the scale of the crisis and, and how it is handled by local governments, for example, uh, have the power to reduce this cost. And uh in, in this context, being able to track the, the different impact of uh policies and, and on different population groups, uh, it's crucial to inform and better, and better structured interventions. Uh, we believe that this work I'm going to present to you today uh makes a step forward in this direction. Uh, already, lots of work has been done in this respect, uh, by leveraging, uh, data from mobile phone sources. Uh, however, we think that, uh, a lot can be done in this field, and, uh, here, uh, are a few gaps we identified. So first of all, most of the work so far we're focused on, uh, high-income countries and we're, uh, we're focused only on small scale like cities and single countries. Additionally, most often the literature in this field makes use of aggregated data and this usually translates uh in the impossibility to follow specific groups of individual and thus, uh, it could, it can provide only general analysis of the population as a whole. Uh, these factors needs enormous literature gap and do not provide easy generalizable results in support, uh, for example, to policymaker working in middle-income countries. In this respect, this work aims at filling this gap and we do so by focusing on mobility and human development. In particular, we focus on 6 middle-income countries which are distributed across 3 different continents to find a most common behavioral response across different regions of the globe and more specifically, we do so by creating a nanodata data set of visited locations. Uh, to dynamically classify each user's home and work locations in an automatic fashion. We then classify each user in terms of the, the wealth using fine-grained census data. And then we took the angle of human mobility behavior with particular attention to the ability of Individuals who self-isolate at home, which is really important to, uh, during a pandemic, and their uh tendency to migrate towards less densely populated areas of the countries and on their committing patterns. This quantity, these quantities in the context of COVID-19 pandemic have, are particularly relevant repercussion in the self-protection dynamic, but also on the job economy and city management which So, uh, and now I, I'm gonna present to you, uh, first the data, then how we process this uh GPS data sourced by mobile phone and how we combined all these data with the information about the wealth and census to, to analyze this different aspect of human mobility behavior. So, our large scale analysis, uh, as I said, this focuses on six countries which are Mexico, Colombia, Brazil, South Africa, Indonesia, and the Philippines, and we're leveraging information from a mobile phone gathered by uh a Veras company which took together thousands of apps and, and, and SDKs all around the world. And in this presentation, I will be showing results only for Mexico, but this is just for space constraints and, and visualization purposes. All the results that can be discussed extend to all the other countries I just listed, listed. So, uh, let, let me stop here about the data and let's dig a bit more into the, the methodology we adopted to handle this type of data. A single GPS trajectory consists of time-ordered sequence of coordinates and, and the data of our disposal consisted of 109 billion GPS points uh from hundreds of millions of different users. To announce the quality of this data, we focus on around 3 million of highly active user trajectories over the entire 2020 period. Given the, the amount of data and the fascination of GPS points, we are interested in filtering irrelevant information and to focus on, focus on one question, we want to, we want to measure one thing which is uh where did people spend their time during 2020. In particular, we are interested in the type of the location which are visited by a user. And the time a user spent in those locations. The information is, to start this information, we construct a 3, 3-step classification procedure. The first one goes from raw GPS points to so-called stop events which are cluster of temporarily and spatially closed uh GPS points. And the second step, uh, cluster together stop events based only on spatial proximity and returns a cluster whose elements are so close to 11 to the other, to another that they are likely to be the same location a user might be visiting multiple times over different periods. The output of the steps providers with the so-called stop locations. And in the third step, uh, based on, uh, the time spent, uh, the, the time spent in each sub-location and the time at which sublocation are visited by the users, uh, we assign a home and work location to, and work location to each individual. Um, it is important to stress that our definition of home and work location is dynamical, and this makes, makes possible for us to study, for example, how people relocated over time during this year. The, uh, entire processing and classification procedure was done as a data set which contained like individual sequences of stop location and each one with a duration plus the location time. We could have stopped here, but instead, we complemented this rule-based classification with an additional optimization step which was intended to uh make the parameters and the threshold uh we choose non-arbitrary. So, uh, this, this step is usually surprisingly overlooked by mobility researcher, uh, even, even though it actually constitutes the very backbone of this research, uh, field. what we did was to create a manually annotated data set consisting of human label location types, having two independent t headrs and asking them to label stop as either on, work, or other location. And, and what they were supposed to do was to um use a dedicated dashboard, which is the one you, you can see here in the slides, and, and they were asked to manually annotate one user at a time, the location of a set of 100 users for each country. Then we use this, the, the output of the data, so these labels to optimize the grid search exploration of the main parameters of our automatic labeling procedure. In summary, the, the entire processing up to this point returns us with a temporary order sequence of labels stop location for each user in each country. What we do more is that we connect uh census data information uh and To create using existing methodologies, a high resolution wealth index. Uh, census data include, among other information, for example, uh, asset ownership by, by individuals and their houses, house characteristics. For each country, we gather, we gather the finest granular data available and we validate for some of these countries the reliability of the wealth index comparing it with values of local poverty measures. And here, for example, in the In the figure, you can see uh the example from Mexico and you can see that there is a very high correlation between our index and the computed index of measure. We measured, we, we computed and their local poverty index. We then decompose the wealthy index into three-tiered class, into a three-tiered class system and assign for each user a wealth status based on the administrative unit where her primary home locations fall in. So, uh, with this methodological background in mind, we can now move to the results of our, of our analysis which, uh, in, in general, we know that socio-economic factors can strongly shape our behavior. Here, what we find is, which is consistent with, with previous studies on mobility, is that in all six countries, there was a significant increment in the fraction of individuals that do not leave their, uh, their house during the pandemic. However, uh, a significant difference is also found in the relative change between uh the high wealth socioeconomic groups and the low wealth socioeconomic group with, uh, the wealthier, more easily self-isolating than the less wealthy. Uh, the pandemic, uh, also prompted people to migrate, making them to leave their houses, for example, and regulate relocate from urban areas to less densely populated areas. Uh, while sometimes for the news, uh, it was reported that a huge migration flow was induced by the pandemic, here we quantify and measure, uh, the, uh, flow from urban areas, the net flow from urban areas to rural ones, and find that on average for each country, around 1% of migrants were present with respect to the country population. By splitting the different groups, we find also that high wealth individuals also in this case, were more prone to relocate during the pandemic. And since uh we find that the most, the, the, the difference in time allocation between high wealth and low wealth group can be associated with different work behaviors, here we, we shift our attention to uh individual commuting patterns and in particular, we focus on the fraction of individual which commutes to their home location as the pandemic unfolds. Uh, to, to, to quantify, to give a sense on, on, on hand on the possibility of a specific group to work from home, for example, but also on the other hand, to quantify the stress that, and that the economic sector in which low wealth or high wealth individuals, for example, are working, uh, are facing over time. We consistently find significant change for all groups in their commuting patterns and more specifically, unsurprising, unsurprisingly high wealth individuals show a relatively higher level of reduction in commuting patterns. While low wealth individuals working in high wealth neighborhoods surprisingly differentiate from low wealth individuals working in low wealth neighborhoods. In particular, what we see is on all the six countries is that they experience. Up to 2 times greater reduction in commuting patterns. And the fact that previous work have found a significantly smaller possibility to work from remote for individuals who live in low-wealth administrative units here suggests that um this eco this socioeconomic group might be facing a stronger economic impact. And it signals that, for example, place-based policies might be a good option for target local intervention. But, from, from a different angle, uh, we can ask whether the enacted policies, has a differentiated impact on individuals uh from different socio-economic groups and if over time there was a reduction or uh a strengthening of this gap. In this respect, we use a statistical model to disentangle the possible determinants of wealth group mobility behavioral changes uh by focusing both on epidemiological indicators as well on the enacted policies. What we do is that we distinguish three different policies groups, the containment, the economic support, and the health policies and use a fixed effect model to capture the inter-country variability and, and uh perform uh MCMC optimization to, to, to perform the regression. What we, we find here is that the containment policies were the most important factor shaping the mobility behavior of all the groups. And in particular, our containment policies had the highest impact on high wealth individuals again. Then, by performing a partial motor recalibration over different windows of time, we find that these results holds over different pandemic periods. Moreover, over time, we find a progressive reduction in the policy relevance for individuals living in low wealth neighborhoods, and this potentially mirrors, for example, a, a gradual differentiation in the policies as the pandemic evolved on different socioeconomic groups. So going back to the, to the bigger picture with this work, we provided evidence of uh common behavioral aspect of six different countries displayed on 3 different continents over the entire 2020. To achieve this goal, what we did was to develop a method a new methodology which makes it possible to use, to dynamically study how mobility behavior patterns changed over time. We differentiated the analysis by grouping uh Mobile phone users, uh, based on, on their socio-economic group of their uh home location and, and found that there is a striking difference in mobility indicators, uh, cross-country which uh with high wealth group, uh, that shows that the high wealth group are more uh prone to self-isolate and work from remote. Second, what we found was that most of the group differences can be explained in terms of difference in work patterns and not other types of locations, with high wealth individuals being more likely to have access to remote workable employment and low wealth individuals with less opportunity to do so. That is also suggests also that the most fragile group can be identified in individuals living in low wealth neighborhoods and working in high wealth neighborhoods. And more in general, the, the low wealth group which as a whole represent the most exposed group to the pandemic threat and these results is also, was also supported and made stronger by the temporal evolution of the impact of containment policies on computing patterns of low wealth individuals. Overall, what we believe is that this analysis and this methodology provide an important evidence for policymakers to focus on and, and a potential valuable tool in support of policy evaluation. In principle, having access to such mobile phone data and such kind of specific sensors. Data might provide both a low cost and a real-time analytical power for most of the countries around the world, facilitating also, for example, better policy design also for those regions which might have less resources to do so. And thanks, I want to thank all the team and thanks to you for inviting me here again. Well, that was fascinating. It makes me feel my age that I barely understood some of what was going on and leave alone the capacity of, uh, uh, to do something that involves 109 billion data points. We were running regressions with, you know, we were lucky 500 data points and uh so cranky version or creaky version of data. Clearly times have moved on, um, but I think, uh, uh, Ariana, um. You tell me what we do now. Do we have the question and answer? Um, OK, so we move to in, in, uh, in-person question and answer, uh, first, and then, uh, anything that comes, uh, from the chat as well. So, uh, as I understand it, um, please be brief. If you have a question, step up to the mic, um, and, uh, and, um, then we'll go around and, and see, uh, which of the presenters it refers to. Thank you for the discussion. I like to talk um. We talk about the. Hard campaign with the level of literacy, will you really have affected this art campaign? Because I, at no point in time did I hear him mentioned. How much of the level of literacy might have influenced those ad campaign, so that would be of interest to me. Is it best to collect a couple of questions first? Yep. I was curious from Arman um how he thinks about the dynamic of engaging with social media, um. And like my son, you know, always reads what he disagrees with and then has a lot to talk about, you know, why he disagrees with what he reads as opposed to just reading what, you know, seemed rational and acceptable and so kind of this kind of dynamic engagement and kind of testing how people actually interact with news, whether they're false or true. Uh, quick follow-up also, um, to Arman is, um, you mentioned at the end a little bit on mechanisms, and I was curious if you're thinking about actually doing any follow-up studies where you try to, for example, um, do, uh, I guess, spend the same um or hold back the information from everyone for the same length of time, even if you're not making any changes to it, to see whether it really is just an effect of the time that it takes for the information to reach the person. Versus if it has to do with the people recognizing that the information is being monitored and related to that I was curious whether you informed them to tell them that the information you're receiving is being monitored or misinformation is removed so they were aware that there was meddling or whether um that was not um not actually recognized and and whether that's something that you're thinking about actually testing again to try to understand what is leading to these changes. Um, hello, everyone. Thank you for the wonderful presentations. This is for Lorenzo. Uh, I was wondering if you consider the mobility differences across the different neighborhoods or even across the different countries, as, for example, some people in the low wealth may rely on public transportation, and some countries were more strict in restricting, for example, uh, public transportation in some of these areas and whether, you know, the availability of infrastructure, availability of services, really, you know, play an important role in this difference that you found. Thank you. Great, so I'm told that that, um, we, we need to give the presenters a chance to answer and then uh those of us who are in the room can move on to, uh, I, I understand there's lunch. So, um, so, uh, we could just go around in, in, in turn, um, maybe I'll start with you since you're in the room, um, and then we move to the other two presenters for their, for their thoughts. Thank you. Awesome. Well, thanks for the great questions. Uh, I'll try to be quick, but happy to talk more offline about all this stuff. I've thought a lot about it. On the dynamics, I mean, so this is where we wish we could study more than just engagement in this context and understand if, you know, changes in people's engagement with uh different content leads to change in actual behavior and especially with the sunshine, you know, because that was all ideas, you're exposing them to more of the misinformation but also With the idea of directly rebooting, rebutting it and so we do not have those measures because we were rushing to get this out during COVID. We have done a bunch of more qualitative work around um what other information was out there and what other types of engagement were there in other media to understand where ours fits in with that and I, I have plenty I can talk about with that. We also asked people, we did survey a population and asked them about trust in different sources like whether it is user generated or Official or um you know, trust in getting this type of information from friends and family and different sources and so we can kind of look a little bit about different trust levels, uh, which isn't to say that someone could say I don't trust this, but they still like listening to it and and rebutting it that we don't. I'll think about if there's a way we can identify that directly in our data because we do see every single piece of content that people get exposed to, so we could see if they get exposed to misinformation, then something else and how, how their patterns look. Uh, it's super interesting. On, on holding back, so that is an obvious follow-up. We're thinking about that. Uh, in the meantime, we do have variation in how long it took to moderate and so that's, uh, I'm, I'm in the process of exploring that because it's, it's tricky to code it at the user level and which is the right moderation. We're usually kind of using their first post, how long did that take? Uh, and there's some results there that maybe suggests it's not really about how long it takes, though the minimum amount of time we have is 15 minutes. That's the fastest. So that's the range we're going to be working in is 15 minutes to 12 hours. Uh, and so a study that could get down to the 0, margin would probably be valuable. And lastly, on did they know we were moderating them? Yes, in the sense that when they made a post, they got a message saying this is being reviewed, we will let you know when it goes live, and then they got a text message saying your post has gone live. Uh, I, I have to double check. I don't think we told them yours was found to be misinformation and we took it out. Uh, so maybe that that piece of information wasn't there, but at least they kind of saw this. If they tried to make a post, they saw that it was moderated. We didn't have variation there. It's, it's colinear with the treatment, yeah. Uh, they would not know. They, no, OK, yeah, you, you didn't know what treatment arm you were in at all, otherwise, and, um, there was a general IRB consent at the beginning for everyone, but it wasn't different based on your treatment arm. Good question. Thanks. Uh, great. So we now go to, um, uh, Victor for any, uh, reflections that you may have on the basis of the questions or anything else, um, and then we move on to Lorenzo. So, so Victor has been stunned into silence. So we'll, we'll move to, uh, uh, Lorenzo. Um, I know you had a question directly to you and, uh, any other reflections you have on the basis of the questions. Victor is there, but you're muted, Victor. Sorry, uh, we're having a technical issue. I, they are muted me now, so Lorenzo must be muted. So let, let me answer the question first, if that's fine, uh, Norbert. Uh, so, the, the social media campaigns in India, um, They, they're very heavy on pictures, right? And, and very low in terms of words, right? Uh, to, to maximize, you know, uh the, the likelihood of reaching uh low literate populations. From a study point of view, um, well, we, we conducted a more or less 2025 question survey, question survey. So these are literate uh individuals that were part of our study in both urban and rural areas, so we can say little about um uh low literate populations in our findings. Um, and, and this tends to be a limitation, right, of social media. Whoever's in social media, uh tends to be uh uh relatively literate, right? And of course that, that also means that uh we need to take additional steps to reach illiterate. Relations such as the examples I was providing towards the end of my 10 minute presentation of using voice, pictures for people to to to tell us a little bit more about themselves even if they don't know how to read. Thanks. Thank you, um, and now, uh, we go to the, uh, final words in this case from Lorenzo. That. Uh, OK. Thank, thanks for the question. It's actually, I believe it's really good questions because uh when we are, when I was speaking about containment policies, what we were doing was actually to include their, uh, different, uh, policies enactment. In particular, what, what we were doing was to include uh the following three, which were school closure policies, workplace closure policies, and restrictions on internal movement. So, we, we did not include any specifically target measure for public transportation means, but uh I, I, I think it's like it's a good point. We could definitely include that and see if there are like some patterns changes and for example, we, we recover some of the, the relevant relative importance between the parameters of low wealth income. Groups and high wealth groups. Uh, however, what, what I think is also that the public consultation means either depend on geographical features and the country we are sitting at, we are studying. So what we might not recover is, uh, a uniform result cross country. But uh we should definitely investigate into this. Thank you. Great. Thank you to everybody um for, for joining this session. It was, it was really quite fascinating.

showAllTranscripts

duration

PT1H9S

scene7File

worldbank/Measuring Dev. 2022 Session 2B

scene7Domain

https://worldbank.scene7.com/

scene7FileAvs

worldbank/Measuring Dev. 2022 Session 2B-AVS

title

Measuring Dev 2022 Session 2B

description

Measuring Dev 2022 Session 2B

showTimestampAndTranscript

yes

sec-spacing

section-none

sec-spacing-bottom

section-none

sec-swoosh

sec-swoosh-type16

style

default, transparent-white-components, heading-center, add-style, bg-primary-blue-70-gradient, bg-dynamic-height

col-xs-12

col-sm-12

col-md-3

col-lg-3

col-xs-12

col-sm-12

col-md-7

col-lg-7

add-style
lp-body-content

Measuring Development 2022: The Role of Mobile Data in Global Development Research — Session 2B: Mobile data and Covid-19 Recording

lp-heading-top-medium

lp-heading-bottom-medium

Measuring Development 2022

https://www.worldbank.org/en/events/2022/05/11/measuring-development-2022-the-role-of-mobile-data-in-global-development-research

Opening Remarks

https://www.worldbank.org/en/news/video/2022/05/20/measuring-development-2022-introduction-and-opening-remarks

Session 1

https://www.worldbank.org/en/news/video/2022/05/20/measuring-development-2022-session-1-recording

Session 2A

https://www.worldbank.org/en/news/video/2022/05/20/measuring-development-2022-session-2a-recording

Session 2B

https://www.worldbank.org/en/news/video/2022/05/20/measuring-development-2022-mobile-data-and-covid-19

State of play lecture

https://www.worldbank.org/en/news/video/2022/05/20/measuring-development-2022-lecture-recording

Session 3

https://www.worldbank.org/en/news/video/2022/05/20/measuring-development-2022-mobile-data-for-transport

Session 4

https://www.worldbank.org/en/news/video/2022/05/20/measuring-development-2022-methods-for-mobile-data

Lightning talks

https://www.worldbank.org/en/news/video/2022/05/20/measuring-development-2022-lightning-talks

col-xs-12

col-sm-12

col-md-2

col-lg-2

sec-spacing

section-none

sec-spacing-bottom

section-padding-between

style

default, video-transcript

everything

https://webapi.worldbank.org/aemsite/everything/search

true

{"facets":[],"orderby":"contentDate desc","select":"*","searchFields":"*","search":"*","filter":"contentType eq 'Video'","top":20,"skip":0}

See Rest of Series

yes

sec-spacing

section-none

sec-spacing-bottom

section-padding-between

style

default, bg-neutrals-15-gradient, remove-border, lp-mini-card-primary

Measuring Development 2022: Mobile data and Covid-19

RELATED

See Rest of Series