Side effects – the blind spot of aid evaluations

By Jørn Holm-Hansen and Henrik Wiig, senior researchers, NIBR International Department

What is the effect of development aid? This question is frequently asked, and not seldom combined with an insinuation that the answer is ‘none’. The sector of development aid internationally has responded by setting up an elaborate apparatus to make sure their activities are evaluated. Scholars, researchers and consultants worldwide are being engaged to document and provide impartial assessments of aid effects and impacts. Despite this, very little knowledge has been produced to answer the question. The reason is that the evaluations generally do not address the issue of unintended effects.

In a recently published study (Norad Evaluation Report 2/2014) we show how the issue of unintended affects is being dealt with in state-of-the-art evaluations. The fact that the Norwegian Agency for Development Cooperation – Norad – itself commissioned the report is reassuring. We went through all the 78 evaluation reports issued by Norad from the 38 evaluations that were carried out for them since 2010. The conclusion is clear: Evaluations mainly check whether the aid activities have the intended effects or not. The obvious possibility that aid activities have unintended effects is hardly taken into consideration, at least not in some depth. This is astounding because addressing unintended effects is one of the requirements clearly formulated by OECD/DAC for evaluating development assistance.
Development aid is given without regard to side effects. Medicines are not.
Credit: Bobbie A. Curtis / Wikimedia Commons

Moreover, development aid is an activity that is likely to produce such effects by the fact that it is taking place in vulnerable and complex settings. In fact, some possible side effects are so probable that most evaluation should discuss them. Nonetheless, it seems as if the slightly desperate pursuit of demonstrable goal achievement has placed possible side effects in the evaluators’ blind spot.

An electronic search using key words like ‘unintended’, ‘unexpected’, unforeseen’, unanticipated’ and the like showed that these word were mentioned in no more than 46 percent of the reports. In addition, if the take closer look at not only how often, but in what ways, unintended effects are being addressed, the picture is even more gloomy. When side effects are mentioned it is most often being done in passing and superficially.

Development aid evaluations are being conducted according to detailed Terms-of-References. In all, 40 percent of Norad’s ToRs require a discussion of possible side effects. Some – i.e. one of four evaluations – mention side effects in one way or another even if the ToR does not ask for it.

Easily predicted side effect: Material goods meant to stimulate certain behaviour – e.g. efficient farming – being seen by target groups as an end in itself.
Credit: Julien Harneis / Wikimedia Commons

One of the questions that should be asked in a large number of evaluations is the following: What is the side effect of development aid taking over the “nice issues”, like education, gender equality and environmental protection? Does it lead to a partial “abdication” on the part of the authorities on these issues? And does aid have the unintended side effect that those societal groups that should have been fighting for the good objectives so to speak are withdraw from “real life” to be engaged by the aid sector in a “parallel aid development reality”? This goes for potentially civil servants and leaders of real civil society organisations. Many of those now making a good living as local aid facilitators might otherwise have set up much-needed real-life businesses.

Another easily predicted side-effect of aid is target groups focusing on the material stimuli they receive to pursue specific goals (e.g. more efficient farming). Instead of seeing these stimuli as means to achieve a goal, they tend to see the acquisition of the stimuli as the goal, which may be fully rational in a setting where long-term planning is close to impossible. Development aid not taking these basic facts into consideration is inefficient.

Back to the initial question: How does development aid work? The answer is clear: We do not know. The reason why is that so far we have been content with trying to identify a small piece of the total picture. In order to get a fuller picture of aid effects and impact we will have to venture beyond checking out the intended effects and include the unintended effects as well.

The report can be downloaded from

Will there be a World Cup in Brazil?

By Einar Braathen
Originally published in the Norwegian newspaper Klassekampen 5 March 2014. Translated to English by Celina Sørbøe

Rio de Janeiro three days before the carnival: Yet another street demonstration this record hot Brazilian summer. Twice as many police as demonstrators in the street. The slogans are: Não Vai Ter Copa – “There won´t be World Cup”– and “No to criminalization of protests”.

Não Vai Ter Copa – “There won´t be World Cup” – sounds like a surreal parole in a football crazy country that has already invested around 10 billion USD on arenas for the tournament which will start on June 12th. Still, the parole has received growing support, and not only on social media and among the activists who have already taken to the streets on numerous occasions during this record hot Brazilian summer. In fact, an increasing percentage of the population as a whole is against the country arranging the World Cup: 10 percent in 2008, 26 percent in June last year, up to a record-high 38 percent this February. The Brazilian government and the International Football Federation (FIFA) therefore fear a repetition of the mass protests that took place during the Confederations’ Cup in June 2013, when over 10 million protesters were in the streets in 400 cities throughout Brazil. They are particularly worried that the host cities will be plagued by violent confrontations. Much indicates that this will be a self-fulfilling prophecy.
Photo: Anne Kjersti Bjørn

The demands from June last year are equally prevalent today, including: a significant improvement of public services within the areas of public transportation, health and education; and no to corrupt politicians, who together with entrepreneurs are getting rich from overpriced and socially destructive construction projects.

The presidency, the congress, and several governors are up for elections this October. They have all proven to be incapable of following up on the promises they gave last year to “listen to the voice of the streets” in order to quell the protests. Instead, the corporate-driven media of the country, led by the Globo imperium, is focusing on the violence and vandalism of certain protesters. The politicians are hoping that repressive measures – such as more police, new laws, and harsher penalties – will quell the tensions and isolate demonstrators. There is much at stake in front of these games.

The public opinion continues to be on the side of the protesters. Not as overwhelmingly as in June last year, but 52 percent of the population still express “support the demonstrations”, according to the poll agency Datafolha. On the other hand, the grand coalition that makes up the federal government was approved by only 39 percent of the people this February, compared to around 60 percent before the mass protests of last year.
“I want FIFA standard buses” Photo: Anne Kjersti Bjørn

The government has presented the following bills for the congress: increased penalties – up to three years imprisonment – for whoever destroys public or private property; prohibiting the use of masks during demonstrations; and obligation to announce demonstrations 24 hours in advance to a series of specified governmental organs. In addition, a proposal has surged that would legalize measures the police has already used on multiple occasions, which is preventive actions – including the arrestment of surveillanced persons before the demonstrations begin.

Romero Juca (PMDB), a parliamentary leader of one of the two largest political parties in the government coalition, does however not feel that the government is going far enough. With enthusiastic support from right wing forces, he has proposed a new “law against terrorism”, which explicitly will strike street protesters accused of using violence. Terrorism is, according to this proposal, the act of “provoking or spreading generalized fear or panic through violating human life or the physical integrity, health or freedom of persons”. The prescribed penalties that are being suggested is from 8 to 20 years if the attack is directed towards public or private property, 15 to 24 years if directed towards a human being, and 24 to 30 years if the act of violence results in death. The proposal is being pushed in order to take effect before the World Cup.

At the same time, a massive police corps is being put together. 150 000 officers from the military police and 20 000 private security guards are preparing in special World Cup battalions, with a budget of around 1 billion USD. Ralf Mutschkle, FIFA´s director of security, is part of the chain of command and in charge of the private security guards. In Rio de Janeiro, the World Cup battalion has its own elite troop. The newspaper O Globo recently portrayed their new armor on the front page with the headline Rio gets “Robocop” against Black Blocks. Black Blocks are the “enemy”- supposedly violent protesters.

Surely, there are anarchist groups that idealize the “right to revolt”, or at least the “right to self-defense”. However, researchers, journalists, and human rights´ activists that have observed the clashes between police and protesters point to the fact that the tumults are almost always triggered by an aggressive police. The law enforcers rarely enter in dialogue with the protesters. The police resorts far too quickly to teargas grenades, rubber bullets and “tasers” (laser guns). On the other hand, the grand majority of protesters are opposed to breaking garbage cans and bank branch windows, which are the most common forms of “revenge violence”.

The violence status from June last year until today is that one person has been killed as a direct result of protesters´ actions – a television photographer who was struck in the head by a fireworks rocket. On the other hand, an unknown number of demonstrators or bystanders have passed away in traffic accidents or similar accidents as they have tried to escape the rubber bullets and tear gas of the police.

Of the many thousands of protesters that have been taken in – during the last demonstration in São Paulo on 21 February, 262 were arrested – the police have only been capable of bringing 27 people to court, in spite of their surveillance cameras and other intelligence. Several hundred police officers have been reported for the use of severe violence in service, but no one has been brought to justice. An especially dangerous development is that the police seems to be actively hindering the work of the media in the field. In São Paulo alone, 57 journalists have become victims of police aggression during protests since June last year, and 107 in the country as a whole, according to the Brazilian Association of Investigative Journalism (Abraji).

Sure, there will be a World Cup in Brazil. The question is: what kind of World Cup, and at what price? Economically, socially, and not at least politically, the Brazilians are paying a price that might be too high. Presumably, Sepp Blatter and FIFA will once again get off too easily, with too high profits. One can however ask: is the international audience willing to accept that the right to demonstrate and other democratic rights are being sacrificed because yet another corrupt mega sports show must go on?

For more on this topic, please visit the web site Brazilian Urban Politics