|
|
|
|
|
|
 |
  |
|
 |
Evaluation of Poverty Research Reviews by A.B. Atkinson The Economics of Non-market Transfers in Less Developed Countries (RPO 676-24, Project Supervisor: Jacques van der Gaag) This project was approved in November 1990, with a budget of $160,000, and the completion report is dated December 1992. The project was led by Jacques van der Gaag and Emmanuel Jimenez, with the latter taking sole responsibility from October 1991. The completion report lists 11 papers resulting from the project, of which 1 has been published in the World Bank Economic Review. The two principal consultants employed were Professor Oded Stark, who is an international expert on the economics of migration, and Professor Donald Cox, who has published a number of articles on the economics of private transfers. Objectives The project sought to describe the extent and pattern of private transfers of cash, goods and services in less developed countries. The earlier review by Cox and Jimenez (World Bank Research Observer, July 1990) had identified the importance of these transfers, and the aim of the present project was to assemble and analyse quantitative information on private transfers. The information was drawn from existing household survey data sets. The analysis was concerned with the motives for private transfers and, in particular, how far they would be "crowded out" by public transfers and other government programs. Here the project aimed to contribute both to theoretical modeling of transfers and to the econometric analysis of transfer behaviour. The authors of the proposal bring out very clearly the significance of private transfers for policy-making in less developed countries. The results of this research project should be of value to Bank operational staff, to governments in less developed countries, and to other researchers in the fields of economic development and public economics. It is the kind of project that yields both background material for operations staff and articles in academic journals. Design and implementation Ex ante reviews. The original proposal was commented on by four reviewers and three of their reports were enclosed with the papers I saw. The reviewers were in general favourable but made a number of good points regarding the planned research. (Reviewer 3 also included a rather good joke.) The two main points made by the reviewers concerned the role of the theoretical part of the project and the feasibility of the empirical research given the data available. Both were taken up by the Research Committee. Both reservations are ones that would have concerned me. The economics of private transfers is indeed an under-researched subject. Some times research questions are neglected because no one has thought of them; in other cases they have not been studied because they are inherently difficult to investigate. Private transfers seems to me to fall into the second of these categories. On the theoretical side, there is little agreement about the appropriate way to model transfers, and theoretical results appear to be highly sensitive to the assumptions made. On the empirical side there are considerable difficulties in measuring transfers and interpreting the observations, even before one comes to the question of attempting to discriminate between different hypotheses. In their response to the reviewers, and revised submission, the authors agreed to increase the empirical part of the proposal and to pay greater attention to the quality of the data. Work program. As far as I can tell from the documents submitted, the project proceeded broadly as proposed in the revised submission. The main departure appears to be in the data sets used. The authors were not able to exploit the panel data and they added data sets other than LSMS. The countries finally covered were Peru, Ghana, Côte d'Ivoire, Phillipines and Colombia. Since I make a number of critical remarks below, I should preface my evaluation of the research output by saying that overall I found it to be most impressive. The authors have produced a substantial volume of research which adds significantly to knowledge in this field. Empirical base. I have reversed the order of the questions, starting with the empirical base and then going on to the methodology, since, in response to the Research Committee, the researchers restructured the project to begin by evaluating the quality of the data. The authors' first stage was to "evaluate thoroughly the quality of the data" on private transfers in the surveys available to them. One of my main criticisms is that this part of the research is not reported in any detail. I appreciate that such research is not easily published in academic journals but it is essential that the validation exercise should be accessible in some form. In the case of Peru, the description of the data is around 1 page in the paper "Motives for ..." and 2 pages in the World Bank Economic Review article. The only validation that is reported is a comparison of transfers received and transfers given. The authors consider it reassuring that they are broadly equal, but there are several reasons why this condition does not need to hold. For example, the valuation in cash of transfers in kind may well be different for donors and recipients. The exclusion of rural households means that the balance is affected if there is a net flow between urban and rural areas. There are households with missing data for transfers who may not be a random drawing of the population. I would have welcomed a systematic discussion of the validation exercise, not least because the findings for different countries are rather different-see Table 1. The present discussion is not easy to follow since the authors appear on occasion to be referring to the number of donors and recipients, which can clearly differ, rather than to the total transfers, which should balance (in a closed economy). It would also have been useful to have some consideration of whether there is an external source of validation. I appreciate the difficulties but the question should be addressed.
I emphasise the significance of validation since there are a priori reasons to expect that private transfers may be difficult to measure. Cox and Jimenez in their paper on Colombia draw attention to the evidence from the PSID in the US which indicates that a survey questionnaire containing a special private transfer module produced a much higher reporting rate than a simple summary question. This must cause one to be cautious about the evidence from the other surveys used. Analytical approach. A number of conclusions are drawn from an econometric analysis of the transfer data. These include:
Some of these conclusions have considerable significance for policy. For example, Cox and Jimenez conclude for Peru that "private transfers from young to old would have been nearly 20 percent higher without social security benefits". This conclusion is quoted in Averting the Old Age Crisis (see (2) below). Given the weight attached to these findings, they need to be scrutinised carefully, and there are a number of questions which arise in my mind when reading the papers:
In fact there are a number of places where the authors place interpretations on their econometric results which seem to go beyond what can reasonably be deduced. For example, they infer the existence of capital market imperfections from the finding that transfer receipts and earnings move in opposite directions over the life-cycle. While I am quite willing to accept that people cannot borrow and lend freely at a specified rate of interest, I do not see that this is the only conclusion to be drawn from the empirical finding. This brings me to the theoretical part of the project. Here, with the benefit of hindsight, it was clearly over-optimistic to suppose that the theoretical models could be closely linked to the econometric work. The theoretical part has been largely independent. This is not however to say that it is without value. Insofar as the theoretical papers have been made possible by the project (cf the remark of Reviewer 3), they represent a significant and stimulating contribution. I read with considerable interest the papers by Stark ("Nonmarket transfers and altruism") and by Bergstrom and Stark (American Economic Review, Papers and Proceedings, 1993). My only regret is that there appears to have been no follow-up of the exchange with the anthropologist, Professor Ortiz, referred to in the "Response to Reviewers' Comments". The paper by Cox and Stark ("Intergenerational Transfers ...") shows how theory may suggest considerations which had not previously been taken into account. Organisation of research. The project appears to have been well managed, as far as I can judge from the materials supplied. In the light of my earlier comment about data validation, it might have been desirable to make further contact with those responsible for the statistics of the countries in question. It is not easy to judge the budget. Viewed in terms of a project submitted to a national research council, then the total amount seems reasonable. The consultants appear to have put in a substantial amount of time. Dissemination The papers have been presented at a variety of conferences, although these have mostly been in advanced countries. Particular reference should be made to the session at the American Economic Association 1993 meetings, which appears to have been a considerable success. It cannot be bad for the Bank to be associated with a session in which 3 Nobel Prize-winners took part. Results and cost-effectiveness As already indicated, I am of the view that the project has added materially to knowledge in a field which is of considerable significance for policy. I feel that it was highly successful in meeting its objectives, although-for the reasons noted above-I would be more cautious than the authors in stating the conclusions. The output seems commensurate with the cost. Income Security for Old Age (RPO 677-45, Project Supervisor: Estelle James) This project was approved in May 1992, with a budget of $100,000, and the completion report is dated January 1995. The project was led by Estelle James. The completion report lists 13 outputs. These include the volume Averting the Old Age Crisis, plus the Summary and Statistical Annex, as well as 2 articles resulting from the project, and 7 Policy Research Working Papers. No fewer than 24 consultants were employed from outside the Bank. I should preface my remarks by noting that I acted as a consultant to the project and commented on the draft of the volume. Objectives The objective of the study, as it evolved (see below), was to analyse the effects of old age security systems (public and private, formal and informal) on the welfare of the elderly and on the working of the economy. It was to make policy recommendations covering Eastern European and OECD countries as well as less developed countries. The relevance of this topic is evident. The elderly population is growing and dependency ratios are increasing. Many countries face fiscal problems of financing public pension programs. Pensions policy affects savings and the working of the capital market, which in turn have implications for the rate of growth. The findings are relevant to the work of economists and policy-makers at many levels, and it is good to see the efforts put into the dissemination (see below). Given that a number of the conclusions and recommendations are controversial, it will be very useful if the existence of Averting the Old Age Crisis leads to a widespread debate. Design and implementation Ex ante reviews. The original proposal (November 1991) contained a detailed chapter outline, which, even if subsequently modified, clearly formed an excellent point of departure. In particular, it provided the basis for reviews by outside researchers. Three reviews were included in the documents supplied, and I consider these to be very useful and constructive. There can be little doubt that they served to sharpen the purpose of a project which in its early stages was rather shapeless. The response by Estelle James when she took over the project (January 1992) showed that the team had revised the plan significantly. The change in title (from "Retirement security" to "Income Security for Old Age") was indicative; and the greater emphasis on informal arrangements (Chapter 2) stemmed from this revision. Work program. In general the program appeared to follow the plan, but there was a significant under-estimate of the amount of time required. The completion report refers to the Department having funded additional time. James and Palacios spent an additional year working on the report. The project ended, as far as research funding is concerned, in June 1993, but-if I understand correctly-the report was not published until October 1994. Given the scale of the task, a more realistic view should perhaps have been taken from the outset about the time period required. Evaluation of the report. The Report is impressive for its coverage. It covers informal arrangements for old age, public pension plans, occupational pensions, and personal pensions, which it draws together in its "multipillar" approach. It covers old age security problems around the world, dealing with "countries with young populations", "young but rapidly aging economies", "gradual transitions in middle age", and "middle-aged countries in trouble". It covers research in a wide variety of fields, including the economics of non-market transfers (see above), studies of lifetime redistribution, regulation of occupational pensions, operating costs of mandatory savings schemes, and simulation of policy reforms. This coverage is achieved at a cost. The findings of research are presented in an attractive but simplified way, and the danger of over-simplification is not always avoided. A good example is the assertion on page 17 that "earnings-related benefits are a poor choice for the public pillar". Is this really true in all circumstances? Or does it make assumptions about the nature of social objectives? Or is it based on a set of assumptions about the working of the economy? A second example is provided by page 175, where it is stated that "To the degree that occupational pension funds have ... directed savings toward productive investments, they have enhanced growth in their countries". This is a statement which cannot be gainsaid but it fails to recognise the criticisms which have been made of pension funds for adopting a "short-term" attitude to the stocks which they hold. If they have acted this way, or if management has merely believed that they would, then it may have discouraged investment, such as that in research and development, and hence reduced the long-term rate of growth. The policy conclusions are those of the authors and are not necessarily the only ones that could be drawn from the research. A number of readers (I am one) would not agree with many of the conclusions drawn. A critic would describe it as a polemic rather than a summary of research. For a study which attaches considerable weight to political considerations, there is remarkably little recognition of the political role of the report. When one looks at the report from the standpoint of a particular group of countries, or one country, it becomes apparent that the argument needs to be modified and qualified. Quite a lot of what is asserted does not apply to a particular country. Again a critic would say that a study directed at everywhere applies nowhere. It may not be possible to employ the same analytical framework irrespective of economic structure or social context. In short, viewed as a research project, this study is subject to a number of criticisms, but then it is not a research project in a conventional sense, a point to which I return in Part II. Dissemination The project is different from the others reviewed here in that it led to a "flag-ship" report, which was launched with considerable publicity at the Madrid annual meeting of the Bank. The publication in glossy format by Oxford University Press now doubt attracted a great deal of attention. The authors report a large number of presentations. These were both to academic/professional audiences (for example, American Economic Association and International Social Security Association) and to policy-makers. The countries in which the report was presented include Argentina, Chile, China, Mexico, Peru, Poland, Sweden and Turkey. Results and cost-effectiveness This study has produced a report which has attracted a lot of attention, and as such must be considered good value for money. Its political role in means that it is probably not sensible to make any assessment purely in terms of research contribution. Response by Estelle James I have read the evaluation of this project by Tony Atkinson and have a few comments: First of all, I completely agree with Professor Atkinson about the tension between research and operationally useful messages in Bank publications. As academic researchers, we are trained to report our findings with all relevant technical apparatus and qualifications-which often makes it difficult reading even for trained economists. In contrast, Bank reports are supposed to eliminate much of this paraphernalia in the interest of accessibility to policymakers and Bank staff. And if the writers don't eliminate it, the editors do. In the preparation of Averting, we gave a lot of thought to this dilemma. In general we tried to handle it by making sure that our messages were justified by the evidence and were not misleading. More specifically, we included "issue briefs" at the end of the book that dealt with some of the more complex issues; indicated to the reader which topics were particularly controversial; suggested additional reading at the end of each chapter; and included an extensive bibliography that presented all points of view. If we had cited all the qualifications mentioned by the reviewer in the box on the Philippines and Peru on p. 67 (incidentally, that box was written by the author of the original source), they would not have changed the basic point we were trying to make-that public transfers crowd out private transfers, but not on a one-for-one basis; however, this would have made the box more complicated and therefore harder to read. Similarly, if we had described in greater detail the sensitivity of poverty rates to equivalence scales, economies of scale, definition of income, etc. (instead of the brief mention we gave these items) this would not have changed the basic conclusion that the old are not disproportionately poor in many countries; indeed they are among the wealthiest in some countries, while poverty among young families is a bigger problem. The academic part of me probably would have liked to include more qualifications as we went along, but this would have made the book less accessible to most policymakers, it would have had to be much longer or to omit other important material, and therefore ultimately I believe it would have been less useful. As Professor Atkinson notes, a book that is about the world will not apply in detail to every country-or maybe not to any particular country. In fact, Averting does not lay out a recipe for any one country to follow. Instead, it lays out a roadmap for thinking about the issue of old age security in a coherent way, with different branches and caveats noted for countries at different stages of development and with several policy options (the size of the public v. the private pillar, the design of the public and private pillars, the speed of pension reform, etc.) for all countries. I believe that Bank staffers working in a wide variety of countries have been very sensitive to country circumstances in applying the messages of Averting. But I also believe-because I have been told so many times-that policymakers appreciate having a document that pulls so much material together for them and that provides them with a systematic framework for analyzing these problems and alternative solutions in their own countries. I am glad that, even if Professor Atkinson does not agree with all our conclusions, he feels Averting will be very useful if it leads to a widespread debate. In my view, a primary objective of a report of this sort is to start that debate, instead of hiding behind an acceptance of the status quo, and to provide a set of issues and facts that must be confronted in the debate. The scores of invitations that other team members and I have gotten to speak at conferences organized by non-Bank groups (universities, labor unions, investment institutions), to meet with government officials, and to prepare supplementary papers, is an indication that this objective has been achieved. (Incidentally if you include the supplementary papers that I wrote after the completion report was submitted, the output of the project has been 14 articles, not 2). Another purpose of such a report is to suggest an agenda for future research-because we never have all the answers. Averting raises several unanswered questions and we are now preparing a research proposal to address them. Finally, it would have been nice (for me personally as well as for the Bank bureaucratically) to estimate the required time more accurately at the beginning, but I can assure you that this would have been impossible-I learned about the scope of the topic as I went along. The extra time improved the quality, and I appreciate the fact that the Bank was willing to make this investment. Management of Drought Risks in Rural Areas (RPO 677-51, Project Supervisor: Jock Anderson) This project was approved in June 1992, with a budget of $98,273, and the completion report is dated August 1994. The project was led by Jock Anderson and Harold Alderman. The completion report lists 2 papers resulting from the project, one circulated as a Policy Research Working Paper. The two principal consultants employed from outside the Bank were Professor Thomas Reardon of the Department of Agricultural Economics, Michigan State University and Dr Peter Hazell of the International Food Policy Research Institute, Washington. I should note that this subject is outside my main field of research and that relatively little information is provided in the completion report. Both limit what I can usefully say. Objectives The project sought to examine the management of drought risks in rural areas and determine the latent demand for improved credit and insurance. It sought to use existing survey data covering the same households for a severe drought and a subsequent normal year. The areas covered are regions in Burkina Faso and South India. The study was intended to contribute to the Bank's work in drought-prone areas in several respects, providing quantitative information about how individuals seek to cope with risk, identifying those most in need of assistance, and indicating ways in which programs could be made more effective. These seem valuable, and the findings should be of value to Bank operational staff (from whom there were letters of support) as well as to other researchers. Design and implementation Ex ante reviews. The original proposal was commented on by four reviewers and their reports were enclosed with the papers I saw. The reports were mixed in the sense that some commented in detail on the proposal or referred to other relevant research, while others made rather general remarks about the field which were of limited assistance to the development of this particular project. Insofar as they focused on the project, the reviewers were in general favourable but made a number of good points regarding the planned research. The response by the authors (May 1992) tended to concentrate on the more detailed aspects. Work program. The Completion report does not comment on the work program but from the papers I deduce that the program was carried out broadly as planned. It would have been helpful to have had more information from the project supervisor. Analytical framework. The Policy Research Working paper provides a good introduction to the analytical setting. The authors argue that much earlier research has concentrated on the supply side of insurance against covariate risk, whereas they wish to examine whether there is a latent demand for insurance by poor rural households. To this end they develop a model based on intertemporal expected utility maximisation, drawing on work in consumer theory. There is a two-stage decision process each year corresponding to the period before and after information is available about the level of rainfall. The demand for insurance is represented by the utility tradeoff between drought and non-drought outcomes, which depends both on attitudes to risk and on the ability of households to absorb risk. If this is greater than the actuarial tradeoff, then, the authors argue, there is unmet demand for insurance. It goes without saying that this is an ambitious undertaking. To derive functional forms which can be estimated one has to make a series of strong assumptions. This applies to the derivation of the household first-order conditions, which then have to be combined with information about the production function and about the income shocks. In my judgment the authors have done an excellent job. It is a distinct achievement to make this approach work when applied to real-world data, and they have exhibited considerable skill in modeling. This does not however mean that one can necessarily believe the resulting estimates. The final conclusion-that the tradeoff indicates unmet demand-may well be extremely sensitive to the assumptions made. Apart from the inclusion or exclusion of village effects, there is very little to guide the reader as to the sensitivity of the findings. Empirical work. The proposal contained an excellent description of the data sets for the two countries. Indeed, I would describe Appendix B as a model of its kind, and it is a shame that it is unlikely to be published. I am not an expert on agricultural or village surveys, but the IFPRI Tamil Nadu data and the ICRISAT data for Burkina Faso seem to me well suited to the purposes of the project. Dissemination One of the papers has been circulated as a Policy Research Working Paper; the other has apparently been presented at the 1994 Annual Meetings of the American Agricultural Economics Association. The completion report says simply that "The policy implications of both the case studies in India and Burkina Faso are still being refined, and dialogue within policy circles in both countries is planned as a further activity during FY'95". Results and cost-effectiveness This seems to me-a non-expert-a very interesting and impressive project. The methods employed mean that the conclusions must at present be treated with caution, but this is just the kind of research which the Bank should be supporting. It is not easy to judge the budget. Viewed in terms of a project submitted to a research council, then the total amount seems reasonable. The consultants appear to have put in a substantial amount of time. The Role and Magnitude of the Private Safety Net During Transition: Private Inter-Household Transfers in Poland (RPO 677-96, Project Supervisor Emmanuel Jimenez) This project was approved in January 1993, with a budget of $40,000, and the completion report is dated April 1994. The project was led by Emmanuel Jimenez. The two principal consultants employed were Professor Donald Cox, who was involved in the first project reviewed above, and Dr Wlodek Okrasa of Warsaw and the Social Science Research Council. Objectives This project was concerned with measuring the extent of private transfers of income between households in Poland and seeking to predict how the pattern of transfers might change. As the authors correctly state, there is good reason to expect informal networks to be prevalent in Eastern European countries, and we know very little about how these function. The project as a whole seems very well motivated. As explained by the authors in the proposal, the pressures on government transfer systems as a result of economic transition mean that alternative forms of provision are of great importance. The redistribution provided by families in an informal manner can in principle be a major complement to public transfers. At the very least it is essential to understand how such transfers function in order to avoid adversely affecting them by policy changes. More ambitiously, private transfers may come to play a more substantial role, thereby reducing the budgetary costs and reducing the incidence of poverty. I am therefore quite persuaded by the value of this research topic for the development of policy. It should be of value to Bank operational staff, to governments in Eastern European countries (and more widely), and to other researchers. A further objective of the project was to obtain for future Bank use the cross-sectional data from the Polish Household Budget Survey and the panel data for the years 1987-1992. Design and implementation The project was able to draw on the earlier project (reviewed above) "The Economics of Non-Market Transfers in Less Developed Countries" directed by Jimenez, applying the experience with research on less developed countries to Poland. A number of my comments there apply also the present project. For this reason, and because the documentation supplied was less extensive, my report on this project is briefer. Ex ante reviews. No reviewers' comments were supplied to me. Work program. The end product consisted of a paper "Family Safety Nets and Economic Transition" by Cox, Jimenez and Okrasa, which covered the original project proposal, and an Annex to a Bank report on the subject of "The Poor Before and During the Transition" (no author specified). I discuss these in turn. The main paper makes use of data from the Household Budget Survey (HBS) to carry out an analysis similar to that for less developed countries described in the first project. Some of the same issues arise. The quality of responses to the questions on private transfers is not discussed, and no balancing calculation can be made in view of the prevalence of transfers from abroad. There is no discussion of the problem of measuring non-monetary receipts. Reference is made to the fact that the value is estimated by both the respondent and the interviewer, but no information is given about differences in valuation. It is surprising that this problem is not addressed in view of the extensive literature on rationing and shortages. Finally, I am surprised too by the observation that the volume of transfers (relative to total disposable income) is the same in Poland as in the US. I would have expected the Polish figure to have been higher. Is this because of the use of a sample restricted to employees? The paper does not discuss in detail the results of Table 1, which contains a lot of interesting material. To give one illustration: on average, the first earner's wages of net donors are only 16% higher than those of net recipients. I would prefer to see more in the way of cross-tabulations before going to the probit/Tobit analysis. Again I would like to see a separation of gross and net flows. Who are the 14 percent who are both donors and recipients? There are also features of the econometric work which seem to warrant discussion: for example, the use of a transformation of age to identify the generalised Tobit (footnote 16). Little explanation is given for the choice of right hand variables. Some, such as car ownership, seem likely to be functions of income. What criterion of statistical significance should be applied with samples of nearly 13,000? The simulation of the impact of earnings loss on private transfers is of considerable interest. At the same time, the details of the simulation are not spelled out. It would be interesting to know what they can say about the source of transfers. The authors write as though private transfers are automatically supplied, whereas, as noted earlier, there may be a limited total of transfers forthcoming from the net donors. The second paper makes use of the panel nature of the HBS data, but is not focused on private transfers. Instead it provides evidence about the persistence of poverty before and after market reforms. The findings are of interest but raise a number of questions. For example, it is necessary to know how the findings are affected by attrition from the panel. No real information is provided to allow one to assess this or other aspects relevant to the quality of the data. For example, the differences between the "low income" and "social minimum" results do not seem to be adequately accounted for. This paper is useful as a first pass at the subject but the temporal dimension seems worthy of more careful study. Dissemination No information is supplied about dissemination. Results and cost-effectiveness The project produced a useful set of preliminary evidence on a subject of considerable importance. Given that a substantial part of the cost was associated with the assembly of the data, it appears to have been good value for money. Overall Evaluation of Bank Research on Poverty Patterns in evaluation The Bank's review and decision-making process appears, from this small sample, to be working well. The reviewers of the three projects for which I saw reports were in most cases constructively critical. They made a number of useful suggestions for the design of the research and these were in general responded to by the researchers. The Research Committee appears to have identified the main issues when requesting amendments. In the case of the Old Age Security project, the review process led to major changes: the final project was much more sharply focussed. Without seeing the projects which were rejected, I cannot comment on whether the Bank chose the best use of its resources, but these four certainly seem in absolute terms a good choice. In the case of the Old Age Security project, there was a significant under-estimate of the time required. In the end this does not appear to have been serious, but this project led me to wonder about the extent of outside monitoring during the life of a project. (By "outside" I mean external to the research team.) With expenditure on this scale, one would expect in other contexts for there to be an external advisory committee, which would meet regularly to review progress. I suggest that this could be considered in the case of large projects. My only substantial comment about the procedure concerns the completion reports. The four varied considerably, and none was fully satisfactory. A lot of the summary information I needed had to be obtained from other papers. I suggest that consideration be given to the form of the report and that research supervisors be told to devote more effort to this final task. As far as the substance of the research is concerned, I suggest that there should be a 1 page summary like those for ongoing projects in Abstracts of Current Research and a fuller summary of the findings like that in the project on the economics of non-market transfers in less developed countries, although even here I had to construct Table 1 below for myself to summarise what they had done. Bank research Reviewing four projects like these provides an opportunity to consider the relation between research in the Bank and academic research. In some respects they are close. Many projects lead to papers which are published in first-rank academic journals. But there are differences. While academics very often have a particular view of the topic which they wish to put across, this is open to challenge on the grounds that it is not justified by the balance of the available evidence. If they often over-state their findings, it is nonetheless the case that academic research can often legitmately be inconclusive. In the case of Bank research, it is directed at the policy and operating concerns of the Bank. People in the field need answers. The Bank wants to put across a message which is clear and unambiguous. It wants to be able to make recommendations like limiting the role of public pensions. But it is evident that such recommendations are not purely matters of research. There is always some degree of tension in research, since even the purest of academics have political views. But these tensions are particularly marked in the case of the Bank, and I am not convinced that they have been adequately recognised-let alone resolved. Two examples may illustrate. The first example brings together the first two projects: that on the economics of non-market transfers in less developed countries and that on old age security. On page 67 of the report Averting the Old Age Crisis, the reader is told that "social security prompts reductions in private transfers". Reference is made to evidence for Peru and for the Phillipines. In the latter case, a dollar increase in public pensions is said to be associated with a decline of 37 cents in private transfers, leaving 63 cents for the elderly. If we now go back to the original source, via the cited reference, we find that
None of these points need invalidate the conclusion drawn, but it is possible that the outcome may be different from that predicted. At the least they should qualify the firmness with which the findings are expressed. The reader of Averting the Old Age Crisis may in the short-term be grateful to be spared such details, but in the long-term, if the policy proposals do not bear the promised fruit, then they may ask why they were not warned. The second example also applies to the project on old age security, and to a number of Bank projects concerned with poverty not reviewed here. One of the bases for the policy recommendations in Averting the Old Age Crisis is the conclusion that the old are not disproportionately poor: "Myth 1" is that old people are poor (page 11). But this depends on how "poor" is defined. There are a number of dimensions to the definition of poverty, as has recently been summarised in a Bank publication by Martin Ravallion. For instance, the Luxembourg Income Study, used on page 79, has shown that the poverty rates are quite sensitive to the choice of equivalence scale. With a scale which gives little weight to children, the relative incidence of poverty among the elderly increases. A scale which gives weight to medical costs or disability will also give greater weight. The results depend on the definition of the unit of assessment. If we take the household, then this may include elderly people as well as families with children. The findings depend on the treatment of capital income. The relative poverty rates depend on the overall level of the poverty line. The Bank's work too often seems to ignore these issues, yet they are not mere academic pedantry. Differences of view exist and some people believe that poverty among the elderly is more important than do others. Broader comments Some of the comments already made are fairly general, and I have suggested some directions for change. In conclusion it may therefore be fairer to emphasise the very substantial contribution which the Bank's research program is making to the fields of development and public economics, and the significance of its publications. Among the features which I commend are the way in which the Bank is exploiting existing household survey data (well illustrated by the present projects), its willingness to apply the findings of empirical research and construct policy-oriented simulation models, and the way in which its researchers are able to take part in the academic community with freedom to publish their findings. Reviews by Michael Lipton Poverty Alleviation and Adjustment in Malaysia (RPO 675-09, Project Supervisor: Homi Kharas) The research aims were to explore, mainly via four household surveys, (i) changes in poverty and its characteristics, in "identification of race with economic function", and in income distribution; (ii) whether changes were due to growth, changing occupational structure, or targeted programmes; (iii) how the changes were affected by the growth slowdown from 1984, and policy during adjustment; and (iv) which programmes were cost-effective against hard-core poverty (Request for Research Funding (hereafter RRF) 675-09). "It involves econometric estimates of wage formation, labour force participation, and sector choice based on micro and macro data" (Abstracts 1990: 89). An unpublished CEM, Report no. 8467-MA of January 1991-stated in its Foreword to be "based on a Bank mission which visited Malaysia from January 8 to January 29 1990"-is nevertheless identified by the Supervisor as the "principal output" of the research. He states that "there was little separation between the research effort and the policy dialogue effort", and commends the project for: advancing Bank policy dialogue ("without the research, the Government would [not] have agreed to let the Bank look at distribution"); improving Government policy in Malaysia; "helping open up a contentious field to public debate"; and reducing academic scepticism-creating "confidence in the findings of official surveys about poverty and distribution" (PCR-RPO 67509: 2-3, 5). There is no way in which an outside reviewer can judge whether or not these policy aims have been achieved. However, the CEM (and the paucity of published results) justifies two referees who warned that the project's vague methodologies would prevent it from achieving the research aims. Also, lack of clarity about the basic data-as shown below-means that we cannot be really confident about many research findings in the project. In that case, we cannot know whether policy (or dialogue) will be advanced if it accepts project's results. Also, they may well not reduce "academic scepticism" about the value of official data on poverty reduction. How far do the main outcomes of this research-tabulations based on "cleaned" data (how?) from the 1973, 1984 and 1987 Household Surveys-confirm or explain the apparently substantial poverty reduction, and income equalization, that accompanied rapid growth in Malaysia?
The authors claim that the huge poverty reduction is genuine, and due mainly to growth, skill acquisition, and shifts of workers out of initial poverty groups (e.g. estate workers, rice and rubber smallholders)-not to targeted programmes, which were seldom cost-effective. This is plausible. But it is not well supported by this research, owing to: overemphasis on racial, and relative neglect of regional, correlates of poverty and inequality; failure to handle admitted measurement problems; and weak integration of findings on poverty, inequality, and human resource variables.
The lesson is not that referees' scepticism about loose project designs or underspecified models is always justified. Many excellent projects start like this; many others completely change the initial methodology, often becoming less rigorous and innovative, but more relevant. The problem here is not the modeling (there isn't much), but inadequate (a) exploration of measurement issues, and of quality
A CEM probably isn't a wise "principal output" for a Research Committee project. The problems appear in this CEM: in its Restricted status (no outside peer review, limited access to potential users); in an Executive Summary that-inevitably, given the aims of a CEM-does not relate to the project's outline or aims of the RRF (pp. 4-5) or the PCR (pp. 3-4, Table 1); and in several CEM chapters unrelated to the project. The Executive Summary, ES (and the CEM) do not have a brief-but surely the research should?-to relate Malaysia's poverty-distribution record to the "East Asian challenge". Indonesia and Thailand, as well as Malaysia (and Korea and Taiwan-and arguably Japan and China), appear to have achieved rapid growth and industrialization with huge reductions in poverty and inequality. State action seems to have helped all these goals (which supported each other) not just by providing far wider access to land and education than in South Asia or Latin America, but also by industrial policy. This, if true, would challenge the view that targeted, especially subsidized, State production or direction must induce huge waste, leakage and rent-seeking. Yet this CEM-and other outputs of this research such as Bhalla and Kharas (1991)-repeat this standard view, as does the shift away from targeted and subsidized interventions by the Malaysian Government. There is at least a set of issues needing to be addressed here. Para. 4 of the ES, and much else in the CEM, identifies restructuring and poverty reduction far too closely with ethnic equalization. Anand proved that by 1970 within-race far exceeded between-race inequality (and poverty causality) in Malaysia. This well-known result goes undiscussed. Perhaps policy dialogue must respect the "politically correct" agenda, on which poverty and inequality in Malaysia were mainly about race; but has this respect unduly constrained research? Paras. 22-28 and the corresponding Chapter 5 concentrate on estimating the success by its own standards-not the economic costs and benefits-of the Malaysian Government's affirmative action for Bumiputeras. While foreshadowed in the RRF (p. 8), this approach is not obviously consistent with the growth and anti-poverty priorities of the World Bank (or of most researchers). The analysis needs to be more critical, examining the implicit economic costs of discrimination. These may fall on poorer, more marginally employable people even in the favoured group, certainly in other poor groups. The researchers note, from the Surveys, that poverty is now more among Indians than among Bumiputeras, but do not discuss implications for the equity of a policy favouring the latter group. Relatedly, racial equality in educational achievement had been achieved by 1987 (para. 3.33), yet the proportion of educational benefits received by the top 10% of educands is higher in Malaysia than in any other ASEAN country (para. 3.40). Save in cases such as South Africa, policy success against poverty and inequity cannot be judged largely by ethnic variables.
If race is overemphasized, the crucial regional issue-raised in para. 5-is relatively neglected. Data and analysis in most of the report (e.g. paras. 13-15; 4.21; cost of poverty removal data in Table 4.5; infant mortality data in table 3.2) are limited to peninsular Malaysia. Often, data for that area are used for all-Malaysia inferences, e.g. about "Malaysia's" success vis-à-vis other countries in reducing poverty incidence (paras. 4.14, 4.17, and tables 4.1-2).Yet poverty, and perhaps lack of access to social services, is much more serious in Sabah and Sarawak-where in 1989 30% of households among these 3.1 mn. persons were below Malaysia's poverty line than among the 14.3 peninsular Malaysians (12%; Table 4.3).
Paras. ES13-14,
The analysis of education (para. 21) advocates privatisation, cost recovery, and improved basic and secondary quality. This reflects the fiscal and competitive stance of general economic strategy required in a CEM, rather than the aims of the project. However, those aims should have been reflected by analysis of ways to improve the impact on poverty, income distribution and market access of the proposals. This is better treated in the analysis of health expenditure (esp. para. 3.29). The discussion of health-sector performance (esp. para. 3.10) is fascinating, but relies too much on secondary official sources, without questioning their quality, coverage, and comparability over time. The poverty impact of recession is interestingly discussed. Yet there is an asymmetry which the authors do not review. Elsewhere they argue that in booms a well-functioning labour market ensures that low-income areas (at least within peninsular Malaysia) gain via migration, and low-income groups via sectoral shifts. Yet in the 1984-7 recession (e.g. para. 2.48) continuing poverty reduction happens via a rise in rural, relative to urban, incomes. Why no factor response? The discussions of cost-effectiveness of anti-poverty actions leave many gaps. FELDA farmland allocations appear to be four times mean farm size (paras. 4.27, 4.38), hardly suggesting an intention to maximise poverty-reducing impact. To assess the rice subsidy (paras. 4.45), we need to know the ratio of its real cost at world prices to the numbers net-disimpoverished (or, better, to the real net fall in poverty severity), and comparable ratios for other measures; these data are not estimated. I could not see the logic or relevance of the presentation on Muda (paras. 4.46-53); relevant Bank work (Bell-Hazell-Slade; OED impact study) is unmentioned. The general message of these sections-that it is inefficient to fight poverty via big subsidies, especially if they increase with farmland, sales, fishing-boats, etc.-is well taken; but the presentation is confusing. Also, the message is not sufficiently separated from scepticism about all targeted anti-poverty interventions, except for some wise warnings on fisheries policy (para. 4.75). The discussion of targeting would have benefited from fuller discussion with Bank specialists (Jimenez, Kanbur). The proposal in para. 4.88 implies not only, as stated, perfect information, but also no disincentive effects on earnings and intra-family transfers. The scoring method in para. 4.87 loses information about who is poor; not only could a weighting system could be devised (4.89), but one can readily estimate predictive equations, before concluding that one cannot target the hard-core poor (para. 4.92). It would have been useful to evaluate (i) self-targeting options, (ii) targeting on indicators not readily altered. Failing these methods, it is excellent to advise an explicit shift to per-capita targeting (para. 4.99(a)). Expenditure per AE, not income per person, is appropriate, however. Also, it should be made clear that such targeting to persons of a poverty intervention runs into standard information and incentive problems. The right approach is to use national household survey and ancillary information to target groups of people, areas, etc. with (fixed) characteristics highly correlated to per-AE outlay poverty.
Chapter 5 shows that government policy-and, more important, the rapid drift out of agriculture (paras. 5.56-7)-has restructured employment so as to eliminate Bumiputera disadvantage. But this is not clearly related to poverty or income distribution. The policy agenda (Ch. 6) is not a research output, but forms part of the policy-dialogue impact claimed in the PCR. Both this impact-if effective-is improved by the research only if the findings are reliable. For example, are Sabah and Sarawak allowed for in saying that "poverty in Malaysia now exists only in small enclaves" (6.9) and that there is "equality of opportunity across races" due to "successful programmes in education" (6.1)? What is the evidence that "commercialization of big land development programs will promote a market for land, which will be a valuable asset for the rural poor"? TOR evaluation guidelines A1-2. The research topics and objectives are critical for development policy, except for assessment of racial restructuring-unless demonstrably part of a poverty-reducing or growth-enhancing process. Intended beneficiaries were mainly research analysts and policymakers; actual main beneficiaries may have been Bank staff in policy dialogue. The researchers' responses-in writing and action-to the reviews were largely defensive. However I don't feel the reviewers spotted the central problems: measurement difficulties and interpretations (despite the high quality of the survey work claimed, probably rightly, in Bhalla and Kharas 1991), and uneasy relationship between research and policy dialogue. I feel the researchers were pushed, or pushed themselves, unduly from the former towards the latter. As a result the research programme was not carried out as planned. The analytical approach consisted mainly of data cleaning (not described), comparative presentation of tables based on successive survey data, analyses of costs and rates of return in social sectors, and a few regressions. Some ready-to-hand improvements are suggested above. The authors used pre-existing household surveys, claimed by them to be of high quality. I believe them, but have no evidence either way. The surveys need recalculations to support consistent, comparable statements about poverty, its trends, correlates and decompositions. Some of these recalculations were not done. On health and education, secondary data from official sources were used; data quality tended to be assumed, not explained. The researchers were of proven quality, yet produced "semi-finished research ... interrupted by ... departure of Bhalla to work on WDR; departure of Kharas on LWOP from World Bank ... [and] Malaysian Government reluctance to accept publication [until] main researchers had moved on" (PCR, p.5). Such events are inevitable, but the Bank might have improved continuity and output by attaching a young, career-stable YP to stick with the project to the finish. Also, the team would have been enriched by consulting a good political scientist about the political aims of, and pressures on, the Malaysian Government regarding its (apparently) anti-poverty programmes. There seems to have been quite close work with local researchers, Government agencies, and some Bank staff; but some key sources and specialists appear not to have been adequately consulted. The decision to "cut one's losses" and integrate the semi-finished findings into a CEM was probably a mistake (see above); a rewrite, an editor and formal publication should have been sought. With updating this is still justified, especially in view of research since 1991 on growth-poverty-inequality relationships. The RRF (items 4 plus 5b) suggests $208,000 as the cost to the Bank. The published research does not justify that sum. The impact on the Government of Malaysia and on the quality of Bank policy dialogue, and (above all) potentially publishable Bank output if completed, may do so. The evidence on poverty, human capital and job/racial/sectoral restructuring supports the findings, which are represented in the conclusions. Weaknesses include too little attention to Sabah-Sarawak, and non-placement of findings in the context of East and SE Asian experiences. Caveats about data are honestly presented, but "too little and too late" and with little effort to overcome them; it is not clear what "data cleaning" comprised. A CEM is not a suitable vehicle for research dissemination, even within the Bank. The writing of the CEM is often hard to follow, and necessary information and definitions often appear scores of pages after results and technical terms. The published paper (Bhalla and Kharas 1991) is much clearer and better-written. However, dissemination beyond the Bank is inadequate, and would warrant expenditure, subject to rewriting to meet the points above, and to some limited updating. The study shows that employment in Malaysia has become ethnically almost neutral, due largely to de-agriculturization; that poverty has fallen sharply, and inequality somewhat; and that this is due much more to growth and education than to targeted programmes. Policy proposals, especially for social-sector financing, are presented to accommodate the new situation. However, all this is subject to serious problems about data, methods, inferences, regional issues, etc. (see above). There is more arithmetical tabulation of survey results, and less statistical testing of hypotheses, than is implied by the RRF.Whether the benefit/cost ratio is adequate depends on (i) a major impact into poverty policy and dialogue, claimed by the researchers, but impossible for outsiders to assess, and anyway valuable only to the extent that the research findings are right; and (ii) dissemination, now the main weakness; much needs to be done to develop and re-order the data for publication, if the project is to be clearly adequate from a benefit-cost viewpoint. Much of the problem was due to disruptions, interruptions, diversions, and some confusion of aims between research and policy dialogue; little of this should be blamed on the researchers. Response by Homi Kharas I would like to make some clarifications and point out some factual errors in the reviewer's report. The reviewer contrasts policy aims with research aims. It was very clear from the outset that the data we were dealing with were NOT publicly available. We did our best to persuade the Malaysian authorities to release the data, unsuccessfully, but did get their permission on a case by case basis to publish some of the findings of the research. In several places, the reviewer questions the quality of the data and says the cleaning process is not mentioned. I do not know what more can be done-we comment on the test with National Income Accounts (which had to be specially derived by us to yield appropriate measures of private disposable income, a fact which is never credited by the reviewer); we tested the distribution of non-respondents by region (which state), and by urban/rural; we matched data in household files with data in member files and aggregated to see if household values corresponded to the sum of individual household member values; we ran regressions with and without observations where some errors were reported to see if it made any significant difference; we compared mean wages from the survey data with mean wages obtained from other published surveys on manufacturing; we looked for zero income and negative income reporting households and household members; we explicitly reject the comparability of the 1957/58 survey using the same methodologies described above (as did Anand 1980, but not Snodgrass). All this is reported in Bhalla 1989. The reviewer chooses to cite one potential inconsistency which we note between mean household income and NIA adjusted figures, and concludes that poverty reduction has been overstated, but without acknowledging either that we mention this fact explicitly (Bhalla, p. 17), nor that we present other comparators that show that the degree of variation with NIA figures (using private consumption) shows a different pattern. In fact, this criticism of the project is particularly inappropriate as a large portion of the funds went into data cleansing, including the recapture of the 1973 survey data which had been lost in both the World Bank and Malaysia. The reviewer suggests using consumption per person as the better measure of poverty than income per person. The data is from Household Income Surveys which do not have expenditure data. The reviewer also suggests that using adult-equivalence measures would have been preferable. Bhalla 1989 explicitly says why we rejected this option-most of the research is about changes in poverty relative to a base year of 1970 where we do not have raw data, only calculations done by Anand in his book. Anand does not use adult-equivalencies, and therefore we cannot either if we wish to maintain a similar methodology. The reviewer asks if household income is converted to a per person basis before poverty calculations and suggests this is left open to the reader. In fact there is a very clear statement of this in Table 4.1 and Table 2.3 of Bhalla and Kharas with two paragraphs devoted to what is used for ranking households in this report, and contrasting this with the then Malaysian preference for ranking by gross household income. The reviewer chastises us for not using internationally comparable figures for Malaysia's poverty line in discussing comparative performance. In fact, he is clearly referring to Table 4.2 Evolution of Poverty-Cross-country Data, which is reproduced directly out of the Poverty WDR (as cited). But we explicitly include a line which we call the "hard-core" poverty line, which is set at 1/2 the official Malaysian line, i.e. Kravis$33/head/month, which is right in the middle of the K$25 to K$58 lines quoted in the Poverty WDR. The reviewer states that poverty levels are underestimated by omitting Sabah and Sarawak from "many" of the discussions. I cannot know what is "many", but we are clear on one point of fact: any discussion of changes in poverty must omit Sabah and Sarawak as there is no data for early (1970 or 1973) years. But let me just quote one sentence (para. 4.19): "Table 4.3 also shows that poverty remains a serious concern in Sabah and Sarawak where almost one-third of households are still poor." The reviewer says that we do not give data for Sabah-Sarawak in Table 4.16 and the subsequent discussion. But this discussion compares our numbers of hard-core poor with the Malaysian government's master list of numbers of hard-core poor which (para. 4.80) is only about Peninsular Malaysia "The Implementation Coordination Unit has constructed a list of hard-core households in Peninsular Malaysia in the following manner." On the question of stochastic dominance and sensitivity of poverty incidence to various poverty levels ("the authors omit to check results for sensitivity to the poverty lines used"), please refer to Bhalla 1989 which shows (Table 6.1) poverty incidence for all years for 6 different poverty lines, and a concomitant discussion. Many of the reviewer's comments are simply too general to respond to: viz. "overemphasis on racial and relative neglect of regional correlates of poverty and inequality"; "weak integration of findings on poverty, inequality and human resource variables"; "problem is not modeling (there isn't much)". This kind of criticism is simply unhelpful. Let me respond by saying: (i) there is inclusion of regional dummies in poverty incidence regressions (and considerable discussion of how regional incomes varied over time and the implications for distribution); (ii) the work on integrating the macro variables on growth (including construction of a capital stock index) with micro variables on labor (age, experience, sex, urbanization, education) to construct a "labor quality" index remains to my knowledge unique; (iii) theoretical work on implications of discrimination for earnings of different groups is very innovative; (iv) identification of female labor force transformation as a principal determinant of poverty reduction and the link with export orientation is of interest. Ad hoc suggestions about what should have been done appear to me to reflect lack of a careful review about what was done, and lack of background knowledge about the relevant issues and basic structure of Malaysia. Poverty and the Social Dimensions of Structural Adjustment in Côte d'Ivoire, 1985-1988-A Policy-oriented Analysis (RPO 675-26, Project Supervisor: Christiaan Grootaert)
The history of this project is fascinating and important for Bank procedures. The initial proposal was rejected (memo from de Tray to Grootaert, Jan. 18, 1989) because the reviewers felt it was not methodologically correct: not "innovative" but "purely descriptive" and too "conceptually straightforward" to be research. Reviewer 3 (Oct. 25 1988) saw no "need for employing expensive talent to do a fairly straightforward task of profiling poverty from LSMS data." Reviewer 4 suggested that only computable general equilibrium models could show how adjustment affected the poor. The general response was that constructing poverty profiles was a low-grade activity, readily carried out by "mere" SDA or Bank staff and not requiring expensive researchers.
The Côte d'Ivoire research "Methodological innovation" did not comprise mainly the 11 April 1989 proposals, of which several (e.g. use of panel data, despite the surveys' odd, tiny rotating panels; pooling of cross-section and time-series observations over four very different years) were not implemented because the research team was flexible enough to see that they had been misconceived, as the reviewers might have inferred from the proposal! The innovations-not predictable when the proposal was written-consisted of decompositions of varying types of poverty indicator over time and space. This allowed the research to identify radical differences between the poverty impact of public-spending changes as between (a) poor and very poor, (b) expenditure and basic-needs indicators, (c) "pure adjustment" years of 1985 and 1986, and the "recession/destabilization" years of 1987 and 1988. Lessons: most good research is serendipitous; the Bank's needs, to find out about poverty and what works or fails against it, overlap badly with the criteria of most economists; so-if you are lucky and find researchers like Grootaert, Kanbur and Pyatt, with a clear problem central to Bank concerns and a good data set-pay them to get on with it, not to write long exchanges with outside reviewers who, though highly competent, are often discipline-rather than problem-oriented. The main research aim was to trace the effects of adjustment on the poor-decomposed into sectoral and regional groups, into poor and "extremely poor", etc.-over four successive years of CILSS surveys (p.5). It was rightly stressed that formal proofs of causality could not be offered without a CGE model, but that designing such a model "requires a good prior understanding of the factual patterns" of poverty (p.8); and indeed the research fully met that research aims by providing this understanding. However, well short of a full CGE model-and, good heavens, economics existed before such models did!-these splendid data structures can be pushed much further. For example, one of the main contributions of the research is to distinguish quite distinct paths (and directions) of poverty change: stasis or improvement during adjustment (1985 and 1986) and sharp worsening during what Grootaert calls "recession/destabilization" (pp. 7, 23, 33, 106, etc.) in 1987 and 1988. This raises two key questions, which the research should try to answer. First, how much of the harm to the poor was due to recession, and how much to destabilization, i.e.(?) to abandonment of stabilization and/or adjustment? On p.23 we are told: "Economic conditions ... continued to deteriorate and this led eventually to an abandonment of the adjustment effort". Second, did adjustment-though helpful to the average living levels of the poor, and especially the poorest, by redistribution via improved rural-urban terms of trade (pp. 57, 59)-increase the poor's vulnerability to falls below these somewhat higher levels, by encouraging specialization and a shift to tradeables production? A series of other results requires more exploration.
This project was in general highly satisfactory, and deserves wide dissemination (but see above points). The main lessons are about Bank review procedures, which seem to have come near to stopping this important work (paras. 26-7).To respond to TOR evaluation guidelines for the project itself: Separating changes of "private consumption poverty" and basic needs satisfaction, for different poverty groups and measures, as between adjustment and recession, is a high policy priority for developing and transition economies. Main audiences are policy analysts and Bank staff in Côte d'Ivoire. Application elsewhere requires reworking of data and poverty lines, as discussed above. There is major usefulness to researchers, as applicability, scope and problems of poverty decompositions have been illuminated. A few well-conceived proposals (to clarify use of part-panel samples; to determine statistical significance for small samples of differences in values of poverty measures; to explore interplay between public-service spending and private-consumption decisions in households) were not sufficiently taken up. In general, however, the researchers responded well-especially in ignoring unwise advice (e.g. to restructure the project around endogenous-choice models of poverty dynamics, health and education outcomes; to do without Warwick support). I can't comment on relevance of research to the memorandum approving the proposal, as this memo wasn't sent me. Grootaert's letter to Birdsall of April 11 1989 seems to respond to such a memo; its emphasis on pooling data across several years, use of panel data to make transition matrices, etc. was NOT reflected in project output-but the decision to abandon this emphasis was right. In general, the researchers responded to their growing knowledge of (a) phasing of events and policy in Côte d'Ivoire, and (b) problems, correction needs, and possibilities of surveys and parallel data. Good research is serendipitous and should be judged by results per unit cost, not by rigid adherence to a work programme. See above. Though the rolling-sample problem wasn't sufficiently recognized, the usefulness of decomposing impacts upon each alpha-measure, of using two poverty lines, etc. was shown. Originality lay less in new procedures than in reworking and adapting data to allow existing procedures to be fruitful: art rather than science, but real research, whatever the academic establishment thinks! See above. Surveys were prior to this project, but it exhaustively, imaginatively and (for once) reproducibly checked, cleaned and allowed for data problems, with a few omissions (homeless "households", etc.). I agree with the authors that, given their starting data and knowledge base, CGE would not have been a useful alternative approach. The authors are too modest in their claims about causal inferences from their results. Team composition (including consultants), management, local collaboration, and advice appear good. There might have been more interaction with other Bank research on poverty (e.g. Ravallion; Kakwani on statistical significance). Reports do not suggest that exogenous shocks were a major problem. The study did, however, uncover a series of errors in the earlier surveys. These were honestly described, and creatively handled. I have not got a complete picture of overheads, but the budget seems to have been tight and well handled. I don't see how results of comparable usefulness could have been obtained more cheaply-certainly not via CGE! Conclusions are consistent with findings, and, both in the main (Grootaert 1992) report and in the technical and other supporting papers, are clearly and well presented, with the necessary caveats. By and large, very good dissemination in a range of refereed journals and in-house and other media. But the main report, while needing a little tidying up, is in several ways an exemplary document and justifies significant outlay to disseminate it as a proper book-not in the "Regional and Sectoral Studies Series", which appears to be rather little reviewed or read. I cannot assess the contribution to analytical capacity in Côte d'Ivoire. Research is problem-solving accumulation of knowledge and understanding. The pre-history of RPO 675-26 suggests that some outside reviewers tend to replace "problem-solving" with "technique-oriented" (para. 27). Main results have been fully discussed above. Regional, group and work-specific levels, and trends, in poverty creatively and valuably explored. Adjustment helped poor to the extent that it addressed anti-rural biases. But harm to poor during "destabilization/recession" needs attribution as between the two. And we need to know: did adjustment, while helping the poor by better rewarding tradeables production, make them more vulnerable because increasingly specialized, and in products exposed to world price fluctuations? Benefits far exceeded costs. A good data base; great care in handling simple but usually neglected issues associated with such data; excellent application of up-to-date poverty measurement and analysis (in most respects); refusal to attempt fashionably over-sophisticated methods which are in these conditions neither robust nor data-compatible. I cannot judge (D-4) Emphatically; see above, esp. paras. 26-27. Response by Christiaan Grootaert My coauthors and I appreciate the thoughtful and detailed review of this project, and we are gratified that the reviewer endorses our methodology and approach in exploring the rich data on Côte d'Ivoire which were available for this project. We share the reviewer's query why the Bank (through its Editorial Committee) has not disseminated this work as a book, but we are pleased to report that the main papers have now been compiled and edited into a book (with the help of a dissemination grant from the Research Committee). This work, as well as a French translation, will be distributed by outside publishers. In general, we agree with the technical comments and suggestions made by the reviewer, several of which would constitute valuable areas for further research An important task of this project was to constitute a user-friendly data base and make it widely available to researchers, and we understand that this data base has been in high demand. We hope that some of the topics proposed by the reviewer for further work will be taken up. We would like to comment on two specific points. The reviewer highlights the project's efforts and success at distinguishing effects on poverty between the adjustment period and the recession years (commenting that "the authors are too modest in their claims about causal inferences from their results"), and recommends further work to differentiate between effects from recession and destabilization (i.e. the abandonment of adjustment). In our data set, the latter did indeed occur simultaneously in the 1987-88 period. A further differentiation requires data over a longer period. In fact, in 1989 the adjustment effort was resumed, leading ultimately to the devaluation of the CFA franc in 1994, while the recession continued (at least until 1995). By comparing this latter period with the years 1987-88, the role of stabilization and adjustment measures could be identified further. The existence of the Priority Surveys of 1993 and 1995 would now make it possible to pursue this issue further. We agree with the reviewer's call for a further analysis of the panel data available for 1985-88. However, the rolling panel nature of the data was such that no household stayed in the panel for longer than two years. It is thus not possible to answer the reviewer's query whether households who stayed in the panel longer fared differently from others. The value of panel data has by now been well demonstrated, in this and many other analyses, and we recommend that the collection of such data be made part of poverty monitoring systems in countries where experience at data collection is adequate. This could usefully be organized by designing a small panel alongside the customary cross-sectional surveys. Indigenous People and Poverty in Latin America: An Empirical Analysis (RPO 677-68, Project Supervisor: George Psacharopoulos) This research is based on surveys in Guatemala, Mexico, Peru and urban Bolivia. It tries to do what the Malaysian report hints at: to estimate the impact of "being indigenous" on poverty, and the possibility of effective policies. The August 1993 report aims to do three things: to estimate whether being indigenous raises the risk of being poor (para. 37); to decompose the characteristics and causes of poverty (para. 38); and to assess how far indigenous people, by raising their education and other productive characteristics to the level of the rest of the population, would reduce their risk of poverty (para. 39). The Review Meeting (see Mr. Cercone's letter of 17 June 1993) rightly recognized the August 1993 draft as "path-breaking", but it is badly damaged by remediable faults: lack of context, methodological flaws, bad presentation. The draft was shown to a commentator for comments before publication. He supplied (a) a letter suggesting the general lines of a revision, and (b) 41 pages of detailed proposals14-to iron out inconsistencies, mistakes and omissions, or to clarify the "story line". Yet the draft appears to have been published with no significant changes. An outstanding piece of research enterprise has been botched in the final stages. It is confirmed that indigenous people are likelier than others to be poor. As is standard Bank procedure, "poor" is correctly defined using poverty line and an extreme-poverty lines that are common across countries and in 1985 PPP dollars per person. Unfortunately, the poverty line (and household resources) are defined via income rather than consumption; the known problems of such a method, and the corrections indicated, are not discussed. Also, poverty lines are set neither at levels reflecting expected dietary energy adequacy (as in the Pakistan study) nor at World Bank levels (Aug. 1993 draft, p.50; WDR 1990, p.29); they are about double these levels, preventing wider comparisons. Even given the definitions and caveats, the results are presented in a diffuse, non-comparable way. Only indirectly, and with needless effort, can one infer a claim such as that "X per cent of the indigenous, and Y per cent of the non-indigenous, are poor in country Z". Also, poverty incidence15 among indigenous and non-indigenous people is not comparable among the surveyed countries, because very different definitions of indigenousness are used, and because the surveys are deficient in non-comparable ways (Bolivia's lasted one month, Peru's two, and Guatemala's was rightly year-long; Bolivia's excludes rural areas; Peru's excludes many of them, containing a quarter of the population; Mexico's excludes questions on indigenousness!).16 These matters probably also damage the country-specific and inter-group poverty estimates and profiles. Some of these things should be put right by going back to the original household data. Where this is infeasible, the effect on poverty estimates (and on other things) should be assessed. The surveys-subject to the above caveats-show that the living conditions, housing, health and schooling of indigenous populations are much worse than those of others; often, the authors show separate data for households above and below the various national poverty lines. However, the justified emphasis on human capital has led to a neglect of many key questions about the demography, nutrition, physical assets (especially land!), employment structure, etc. of indigenous and other households above or below the poverty lines. For example, only for urban Bolivia do we get comparison of household size between poor and non-poor, and in general the family-cycle aspects of poverty (and indigenousness) are not explored.17 The incomplete poverty/indigenousness profiles make it more serious that the contexts are ignored (except for a rather static, ethnically-generalising page, p.104, of Guatemalan history and a discussion, pp.141-3, of the geography of indigenousness in Mexico). Where are the poor, where are the indigenous, and how did their present conditions arise? Most people's "stylised facts" about the Latin American indigenous poor mix up terrible historical realities, geographical adaptations, and caricatures from Left and Right: rural dispossession, isolation and marginalization, enforcing near-landlessness and abusive labour relations; urban family breakdown, child labour and prostitution; high vulnerability to "imported" diseases and drugs (including liquor); stunting, high fertility and child mortality, rapid population growth, and high child/adult ratios,18 all in part costly adaptations to altitude and hunger. This work seldom quantifies whether any of this applies to the indigenous poor. Yet the answers-not only the statistical relationships19 in this book-determine whether (as implied here) better access to education for the indigenous poor is feasible, or likely to delink indigenousness from poverty. It is claimed that "in Bolivia ... the earnings differential between indigenous and non-indigenous working males would narrow by 72 per cent if each group were endowed with the same productive characteristics"; in the other three countries analysed, Guatemala, Mexico and Peru, the narrowing is by 50 per cent. The remaining 50 per cent (in Bolivia 28 per cent) of the differential "reflects both measurement error and ... factors such as disparities in ability, quality of education, labor force participation, culture and labor market discrimination" and is therefore an upper bound to the effect of discrimination (p.xiv). For females, the role of "productive characteristics" is even higher (pp. 89-90, 138, 231). However, the lack of context (para. 38) makes it difficult to draw policy conclusions. The authors imply that equalising education between indigenous and other people (a) is feasible on its own, and (b) would greatly reduce poverty even if nothing happened to discrimination or other matters. As for (a), does landlessness rurally, and the job discrimination in towns, face the indigenous poor with perceived risks to survival, for want of income from child labour, if their children delayed entering the labour force? As for (b)-apart from the absence from this research of a labour market and hence of estimates for how wage-rates might fall in response to a bigger supply of educated labour-if the stylised facts at the end of para. 37 are even half-true, indigenous people from poor households or areas could well get far lower returns to education than indigenous people with better complementary resources. To show that half the income gap between the indigenous and others is statistically associated with educational gaps is NOT to show that the educational gaps can be independently closed; or that if they are closed the average income of indigenous people must rise a lot; or that if it does the poor will benefit. Like the authors, I believe much of this; but I am not sure that they have proved it. This is "a brilliant first draft ... at least two drafts away from publication" (Annex A). There are numerous recurring statistical problems,20 lack of comparability, and unclear units and definitions (e.g. of employment, p.66, and enrolment, table 5.29-30); very opaque presentation, e.g. few tables comparing results across countries; and, most seriously, failure to bring out a clear story line ("the primary focus is descriptive"-p.225-but the country chapters describe different things in different ways and sequences). There is no structured account of the causes of poverty among indigenous Latin Americans, or of the ways to correct it-though there are numerous fascinating bits of analysis and testing. This research cost the Bank only $70,000 (memo from Segura to Ingram, 10 Sep 92 and PCR). Had this purchased only the draft of August 1993 for use as a first shot at a revised draft, it would have been an excellent return. Alas, the draft was published unrevised, "with all its sins upon its head". This project-like the Ivory Coast research, but unlike that on Pakistan and Malaysia-has produced extensive published work in widely-read academic outlets. This enhances the returns. However, for these to be adequate, the revisions indicated above (and in Annex A) still need to be done, and widely disseminated. It would pay to invest, say, a further $30-40,000 in this. I pass now to comments on the aspects outlined on p.2 of the TOR. This research examines the extent to which (a) indigenous (or other) ethnic groups are more "deprived" than others, (b) this is a major correlate or cause of poverty, misery, or inefficiency, and (c) this can be corrected via improved access-by the indigenous, the poor, or both-to human capital. The answers are highly relevant to policy in most developing and post-socialist countries. The main audience is Bank regional staff, policy analysts and senior administrators (and, one hopes, officials in NGOs working for or with indigenous and poor groups) in Latin America. Academics and researchers are a second audience; a third comprises analysts and policymakers working with analogous problems elsewhere. All these audiences would be hugely helped by action along the lines of para. 41 above-to correct the problems with the work in present form; to bring out a clear story line; and to show why different policies regarding the "human capital" of the indigenous poor were more, or less, successful in countering poverty in different situations. Emphasis should be on total impact of human-capital options on the indigenous and other poor, rather than on regressions predicting partial-equilibrium effect on expected earnings of groups at their mean income per person. See Ingram to Psacharopoulos, Sep 29, 1992. Presumably this small project was felt not to require a formal external review procedure. The work programme outlined in the research funding request (Segura to Ingram, Sep 10, 1992. pp.2-3) was partly followed. Income differentials-between rich and poor, indigenous and other-were statistically linked to "human capital versus other variables", but for reasons discussed above it is doubtful that a "due to" relationship was established. "Adding urban and occupational dimensions" was only partly feasible; occupational distributions were not presented in most cases; the Bolivian survey was only, and the Peruvian survey disproportionately, urban. The proposed work on surveys from Paraguay, Brazil and other countries did not prove feasible. On the other hand, the August 1993 draft rightly uses the database to attempt valuable work beyond the research funding request, notably by seeking to identify poverty profiles in the four countries studied in detail. The construction of poverty measures and profiles is not innovative, but it is both important and demanding of creative, intelligent economics. As in the Malaysia study (but in sharp contrast to the Côte d'Ivoire work), several aspects of "best practice" were not observed.21 The other main research task-attribution of income differences among causes, isolating the contribution of human capital-is familiar from the literature, partly due to the work of one of the principal authors (Psacharopoulos). The difficulties are also familiar: "other things equal" assumptions, causal structuring and direction, effect of education and poverty on participation and unemployment, etc. Much modern work tries to deal with these problems. This work did not fully confront them.
The authors should not be criticized for the limitations of the survey materials that they had to use. However, these limitations should have been corrected from primary data if possible, and analysed for impact on results otherwise. To do so would have helped this research to confront preconceptions (about causes and effects of "indigenous poverty in Latin America") that are relevant to the hope that education is feasible, or sufficient, as a weapon against it. The authors rightly avoided full-scale simultaneous-equation models, which are inappropriate to these data. But informal economic logic, applied to the total issue "To what extent is extra education feasible and effective against indigenous poverty?", is essential to avoid implying that this question is answered by regression equations using years-of-schooling to predict indigenous and non-indigenous earnings at the mean. A few hundred dollars for advice from an anthropologist and a historian, both in project design and before publication, would have been well spent. No obvious problems here (B6). The budget sought and granted was sufficient for the work done, but not for the work that should have been proposed and done. The question, "Can education cure indigenous poverty?" was too narrowly defined, and therefore seemed soluble within the budget. There are serious problems (C1, see above). The report shows that at least half the earnings differential between the average indigenous and non-indigenous person is associated with differences in "productive characteristics", i.e. mainly education. It does not follow that supplying equal education is (i) feasible, (ii) likely to cause average indigenous incomes to rise at least halfway to average non-indigenous levels, (iii) likely to affect the well-being (or even income) of poor indigenous (or non-indigenous) people as substantially as it affects the average earnings of all indigenous people. Yet numerous passages in this report tend to lead the reader towards such conclusions. One example: does extreme land inequality, enforcing often abusive labour on poor indigenous children, affect the feasibility or impact of education to reduce their poverty? The August 1993 draft is well written. However, it has no clear story line, and the component chapters are not similar in design or content. The chapters on data and conclusions do not pull the information together neatly or comparably. All these things are readily remediable. Dissemination so far good-too good; the work is path-breaking and the issues are vital, but the main report should not have been published by the Bank before the matters discussed here and in earlier memos had been put right. No comment except as in para. 41 above. Main results are that indigenous poverty substantially exceed that of others in Latin America; that this is linked in important ways with poverty profiles; and that this may be substantially improvable by better access to education (see, however, above). Stated objectives fulfilled; the research plan made clear that there was a calculated risk in leaping from findings about relationships at the mean, to conclusions about cause and effect at the lower tail of the income distribution. This calculated risk is probably justified, but the caveats need full exploring-and relaxing if possible. Benefits will amply justify modest costs, if further smallish sums are put into correcting the problems identified, and distributing the results. The strengths and weaknesses are those of (i) earnings-function approaches and (ii) the basic survey data. Interestingly, two very different sorts of people agree in warning against excessive faith in these earnings functions, at least as causal statements: researchers at the frontiers of economics (who tend to go for simultaneous-equation models); and contextual, multidisciplinary, probably insufficiently rigorous people like me (who, while doubtful that CGE is appropriate or interpretable, want an informal look at the societal structures). I can't assess from the reports, but from names and procedures looks good. Referees before research may favour their own methods and discipline-oriented preconceptions, but serious comments during and after research should be taken seriously. If that lesson is learned, other lessons may look after themselves. Poverty, Growth and Adjustment in Pakistan (RPO 675-29, Project Supervisor: Jacques van der Gaag) This project confronts the reviewer with the same problem as the Malaysia project (sec. A), but in more acute form. This time, not only is there little refereed output, but also there is no final report, and only a very incomplete PCR (of 1 Aug 1995). As with the Malaysia study, the claim is made that the benefits are mainly to policy dialogue, and that the project should not be judged mainly on its research contribution. However, the Bank (and Research Committee) finances the projects as research. Also, it is hard to see how they can make a big contribution to policy dialogue on poverty, unless they are also, through research, finding out new things. In any case, a reviewer can comment only on the research outputs presented. These are interesting but (a) not commensurate with the $139,460 of Research Committee funds spent (PCR, p.5) plus $37,200 of departmental money for Part A alone, (b) not published in ways or places likely to reach many of the policymakers or researchers in the audience. The PCR (p.2) recaps the "questions motivating the work" in the RRF of Jan. 27, 1989: "Has the incidence of poverty fallen as a result of ... growth? What ... actions have been undertaken ... to improve the status of the poor [and] have they been ... successful? In the light of constraints imposed by the new adjustment program, how can ... existing poverty alleviation programs be more effective? What new programs might be needed in order to mitigate the adverse impacts of adjustment on the poor?" The project output is described in the PCR as "three major papers", viz. Deaton and Grimard, Demand Analysis for Tax Reform in Pakistan, LSMS Working Paper no. 85 (but see para. 48 below), hereafter DG; and two unpublished research papers of the Applied Economics Research Centre, University of Karachi, "Poverty in Karachi: Incidence and characteristics" (Altaf, Ercelawn, Rahim; some findings appear in their paper in Pakistan Development Review, Summer 1993), hereafter PK, and "Absolute poverty as risk of hunger: norms, incidence and intensity for rural and urban Pakistan", hereafter AP. To this one should add a third AERC mimeo, Ercelawn's "Poverty trends in Pakistan" (Nov. 1992), hereafter PTP. As one would expect, DG is a splendid paper, methodologically innovative and very clear. However, it does not address the questions defined by the project (para. 47), and the authors acknowledge other sources of research support, not the World Bank; they do not attribute the paper to project RPO-675-29. They show how indirect tax reforms might affect various groups, given carefully specified demand structures, plus survey evidence on own-and cross-elasticities of demand (the latter turn out to be unexpectedly important, reducing or reversing the apparent impact of some tax reforms on tax-efficiency and/or equity: p.32). The authors repeatedly emphasize that "these policy implications do not apply directly to the tax instruments currently in use in Pakistan" nor therefore to tax reforms in the context of adjustment (p.35). The paper, while outstanding as economic research, is not, and is not presented as, part of project output or a contribution to project aims: "Our findings suggest that raising revenue from an increase in the price of rice to consumers would be desirable [for] efficiency and equity. But the main [policy instrument] is the export tax, a decrease in which would ... increase the price of rice, but ... decrease ... revenue" (p.35). The main liberalizing tax reform is not modeled, though analytical tools are developed.
Thus:
The Ercelawn papers are intermediate between the excellent Côte d'Ivoire report and the less-satisfactory Malaysian and Latin American research, in respect of their methodology of poverty measurement, comparison, attribution and profiling. Positively, they rightly use expenditure rather than income, and adult-equivalents rather than persons, as a unit of receipt. They define poverty with a clear anchor-i.e. the poverty (extreme poverty) line is the level of expenditure per adult equivalent at which it is expected that all (80-90 per cent of) dietary energy needs will be met-though that anchor is not related to the standard World Bank (WDR 1990) poverty lines. Kakwani's significance tests are used. Negatively, there is no sensitivity testing of the results to the poverty lines. Also, while these papers go beyond poverty incidence, the second-order measure is Sen's HI instead of alpha-one, and alpha-two is not used. The main innovation, which parallels work by Foster, Greer and Thorbecke in Kenya, is to present estimates of poverty trends and changes that allow for local differences in prices and consumption bundles. On this basis, poverty-line expenditure per month per adult equivalent in 1984-5 is set at R.355 in cities, R. 245 in towns and R.185 in rural areas. Whereas the incidences of rural, town and urban poverty using a uniform national poverty line are respectively 20, 17 and 5 per cent-the corresponding incidences are 18, 37 and 42 per cent using location-specific prices, consumption bundles and hence poverty lines (AP, pp. 18, 32). Ercelawn clearly and honestly sets out the assumptions and reasoning, leading to his conclusion that the risk of absolute poverty in a Pakistani city is double the risk in rural areas. However, even if (as here, and as in standard World Bank procedures) poverty is defined strictly in terms of private consumption, the conclusion is not credible-e.g. from inspection of age-specific mortality, housing conditions, etc.; or from the observation that there is not a sudden rush of city-dwellers to the countryside! The main problem, as Ercelawn recognises, is that the much higher city poverty line reflects not only (i) somewhat higher prices and (ii) a consumption-bundle necessarily containing a higher proportion of non-food items (paid-for housing and sometimes water, fares to and from work), but also (iii) voluntary responses to wider availabil | ||||||||||||||||||||||||||||||||||||||||||||||||