A number of questions on M&E and on building government M&E systems commonly arise at international conferences and seminars. Officials, academics, consultants, donors, and others interested in strengthening government M&E systems often raise similar issues about how to do this and how to overcome the perceived challenges. Answers to 21 of the most frequently asked questions are provided here. The frequency with which similar topics are raised helps identify key issues that must be addressed when seeking to institutionalize an M&E system. These issues also help highlight the many related dimensions of a government system for M&E and clarify the nature of trade-offs that may need to be made.

A fuller discussion of these issues is provided in How to Build M&E Systems to Support Better Government. The chapter references provided below relate to this book. Additional links are provided to other resource materials available from this website on Building Government M&E Systems.

Weak demand is a serious obstacle to building a gov ernment M&E system, but there are a number of steps that can be taken to strengthen demand (see Chapter 10: Building Government M&E Systems—Lessons from Experience and Chapter 11: Incentives for M&E—How to Create Demand). First, it can be helpful to raise awareness of M&E among senior officials: explain what M&E is and the many ways it can be valuable to a government—such as to achieve more informed decision making in the national budget or to improve ongoing management of all government activities. Senior officials can often be persuaded by examples of influential evaluations and well-performing M&E systems in other countries (Chapter 3: Contribution of M&E to Sound Governance). Finding powerful champions and allies is also important, and they in turn can help advocate and support M&E more widely.

Every government is already conducting some M&E work, even if it only comprises ministries' systems of administrative records, the sector data of national statistics agencies, and the information governments provide to donors. It can be eye-opening to conduct a rapid diagnosis of current M&E activities and to discover the extent of duplication and inefficiencies in existing monitoring systems. This can provide the impetus to streamline existing systems and ensure they provide information that is more useful to the government, as happened in Uganda (Chapter 12: Importance of Country Diagnosis).

See also:
Influential Evaluations
Diagnostic Guides

This is difficult to achieve, particularly for civil servants who have spent the whole of their careers conforming to rigid rules and procedures. It is easy for civil servants and their managers to focus only on activities or processes, such as the number of health clinics constructed or the number of checks paid to welfare recipients. But ideally they would focus on the goals and objectives of these activities, and specifically on their results—the outputs, outcomes, and impacts of their efforts.

There is no simple answer to this issue, but there are a number of steps that can help change the mindset of civil servants. One is the introduction of committed, reform-oriented champions who occupy powerful positions in the government. Another is the use of powerful incentives to encourage a greater focus on results and a greater client orientation—a service culture (see, for example, Chapter 11: Incentives for M&E—How to Create Demand). Conducting regular "How Are We Doing" team meetings, for managers and their staff, can help clarify objectives, current team performance, and ways to improve it.

See the answer to Question 2. Your government is already devoting a lot of effort to M&E, especially the monitoring efforts of your ministries and the achievement of donor evaluation requirements. For example, a diagnosis in Uganda discovered 16 monitoring subsystems; another diagnosis in the social development agency in Mexico discovered eight uncoordinated monitoring systems in this one agency.

There is a myth that one percent of a program's spending should be devoted to evaluation. Although there is no logical or empirical basis for this number, it is indicative of the scale of effort and resources that should be devoted to evaluating any activity. If a program is evaluated every three to five years, then even a cost of one percent is equivalent to only one-third to one-fifth of one percent on an annualized basis. Thus, the evaluation would need to result in only a very modest improvement in the effectiveness or efficiency of the program for the evaluation to be cost-effective. Some of the evaluations highlighted in this volume have been much more cost-effective than this.

The cost of an evaluation will be substantially lower if there already exist good monitoring data, especially on program beneficiaries and program outcomes. This is an argument for establishing sound administrative data systems and for investing in national statistical systems.

Of course, evaluations should never be conducted for their own sake; the bottom-line measure of success for any evaluation is that it is used intensively. Lessons on how to ensure that evaluations are influential are provided in Annex A of How to Build M&E Systems to Support Better Government.

Prioritizing evaluations is a common issue for the managers of an established M&E system. Standard criteria used to help select programs for evaluation include the following: large spending programs; programs of particular policy importance; programs that have suspected major problems with their performance; and pilot programs the government is considering scaling up. Additional criteria to prioritize evaluations include the feasibility of conducting the evaluation, whether the evaluation will provide timely information (such as whether the evaluation findings will be available in time to influence the government's decisions), and whether there are enough resources (funds, staff, and so on) to conduct the evaluation.

As always, when deciding which programs to evaluate, it is a good idea to consider questions such as who needs what evaluative information, for what purposes, and when.

For M&E systems that are not yet established, the challenge is to make every evaluation count—to ensure that they will have high visibility to senior decision makers and will also be highly influential. It is helpful in this context to try to anticipate important decision points, such as a planned major review of social spending, an incoming government, or preparation of a new national plan. If these decision points can be anticipated sufficiently far in advance, it should be possible to ensure that the evaluation's findings will be available in time to feed into the government's decision –making.

It is true that very few evaluations are commissioned and paid for by African countries, for example. Although evaluations do not have to be expensive—as outlined in Chapter 2: What Is M&E?—An M&E Primer, rapid reviews and similar types of evaluation can be conducted quite cheaply, costing even just a few thousand dollars—many evaluations are much more expensive than that. There does appear to be a public good argument to have evaluations in poor countries funded by international donors, partly because of the poverty of the countries and partly because the findings from evaluations may well be of benefit to other countries in the Region.

In addition to funding, donors can also bring their technical expertise in evaluation. The downside, however, may well be a much lower level of government ownership of the evaluation findings if senior officials have played no role in the choice of program to evaluate, the management of the evaluation, or its funding. This provides a strong argument for a collegiate approach to evaluation, involving a government-donor partnership.

This is a highly controversial issue. Some proponents of rigorous impact evaluations do appear to argue this. It is certainly the case that a randomized control trial, if performed well, can provide strong evidence of the impacts of a program; this is the standard experimental method for measuring the effects of new medicines, for example. A serious limitation of such rigorous impact evaluation methods, however, is that it is often not possible to apply them to sector-wide or nationwide interventions. These evaluation methods are also typically very expensive, which makes it harder for governments to fund them. Chile, for example, conducts a limited number of impact evaluations each year, while it also conducts a greater number of inexpensive desk reviews (Chapter 6: Chile).

In Latin American countries it is usual for evaluations to be commissioned by government departments but conducted externally. The advantage of this approach is that it avoids a potential conflict of interest. It helps to ensure that the evaluations are more objective and "independent" than if they had been conducted within the government. This increases their perceived credibility and reliability.

Chile's M&E system has experienced the downside this approach can have—the ministries whose programs are being evaluated generally have little "ownership" of the evaluation findings and thus do not make much use of them. Colombia has combined the advantages of external evaluations with the advantages of sector ministry ownership; they are full partners in commissioning and managing the external evaluations (Chapter 7: Colombia).

In OECD countries, it is more usual for evaluations to be conducted internally; this helps ensure that the evaluations draw on the program expertise of the agency's staff. It also encourages agencies to use the evaluations. In Australia, sector ministries evaluated their own programs, but the central ministries would usually play some role in overseeing the evaluations and reviewing the evaluation reports (Chapter 8: Australia).

See also:
Country Case Studies