Side effects – the blind spot of aid evaluations

By Jørn Holm-Hansen and Henrik Wiig, senior researchers, NIBR International Department

What is the effect of development aid? This question is frequently asked, and not seldom combined with an insinuation that the answer is ‘none’. The sector of development aid internationally has responded by setting up an elaborate apparatus to make sure their activities are evaluated. Scholars, researchers and consultants worldwide are being engaged to document and provide impartial assessments of aid effects and impacts. Despite this, very little knowledge has been produced to answer the question. The reason is that the evaluations generally do not address the issue of unintended effects.

In a recently published study (Norad Evaluation Report 2/2014) we show how the issue of unintended affects is being dealt with in state-of-the-art evaluations. The fact that the Norwegian Agency for Development Cooperation – Norad – itself commissioned the report is reassuring. We went through all the 78 evaluation reports issued by Norad from the 38 evaluations that were carried out for them since 2010. The conclusion is clear: Evaluations mainly check whether the aid activities have the intended effects or not. The obvious possibility that aid activities have unintended effects is hardly taken into consideration, at least not in some depth. This is astounding because addressing unintended effects is one of the requirements clearly formulated by OECD/DAC for evaluating development assistance.
Development aid is given without regard to side effects. Medicines are not.
Credit: Bobbie A. Curtis / Wikimedia Commons

Moreover, development aid is an activity that is likely to produce such effects by the fact that it is taking place in vulnerable and complex settings. In fact, some possible side effects are so probable that most evaluation should discuss them. Nonetheless, it seems as if the slightly desperate pursuit of demonstrable goal achievement has placed possible side effects in the evaluators’ blind spot.

An electronic search using key words like ‘unintended’, ‘unexpected’, unforeseen’, unanticipated’ and the like showed that these word were mentioned in no more than 46 percent of the reports. In addition, if the take closer look at not only how often, but in what ways, unintended effects are being addressed, the picture is even more gloomy. When side effects are mentioned it is most often being done in passing and superficially.

Development aid evaluations are being conducted according to detailed Terms-of-References. In all, 40 percent of Norad’s ToRs require a discussion of possible side effects. Some – i.e. one of four evaluations – mention side effects in one way or another even if the ToR does not ask for it.

Easily predicted side effect: Material goods meant to stimulate certain behaviour – e.g. efficient farming – being seen by target groups as an end in itself.
Credit: Julien Harneis / Wikimedia Commons

One of the questions that should be asked in a large number of evaluations is the following: What is the side effect of development aid taking over the “nice issues”, like education, gender equality and environmental protection? Does it lead to a partial “abdication” on the part of the authorities on these issues? And does aid have the unintended side effect that those societal groups that should have been fighting for the good objectives so to speak are withdraw from “real life” to be engaged by the aid sector in a “parallel aid development reality”? This goes for potentially civil servants and leaders of real civil society organisations. Many of those now making a good living as local aid facilitators might otherwise have set up much-needed real-life businesses.

Another easily predicted side-effect of aid is target groups focusing on the material stimuli they receive to pursue specific goals (e.g. more efficient farming). Instead of seeing these stimuli as means to achieve a goal, they tend to see the acquisition of the stimuli as the goal, which may be fully rational in a setting where long-term planning is close to impossible. Development aid not taking these basic facts into consideration is inefficient.

Back to the initial question: How does development aid work? The answer is clear: We do not know. The reason why is that so far we have been content with trying to identify a small piece of the total picture. In order to get a fuller picture of aid effects and impact we will have to venture beyond checking out the intended effects and include the unintended effects as well.

The report can be downloaded from Norad.no.

Leave a Reply

Your email address will not be published. Required fields are marked *