Regaining Perspective: The Debate over Quality Assurance and Accountability

Over the past two years, there has been persistent and increasing opposition to the Sphere and Ombudsman projects, and by extension to field-based or general codes of conduct, including the People-in-Aid code. This opposition has come primarily from French NGOs associated with the Groupe Urgence- Réhabilitation-Développement (Groupe URD). The initial objection to Sphere – that ‘to every complex problem there is a simple solution, and it is a bad one’ – has led to an advice to the French government not to co-fund the Ombudsman project; and has resulted in the creation of a ‘Platform for Quality’ and consideration of a ‘Project for Quality’, in explicit opposition to Sphere and the Ombudsman project (renamed the Humanitarian Accountability Project). The increasingly acrimonious tone of the debate and its apparent polarisation are partially due to the fact that our francophone colleagues feel they are not being heard and taken seriously. That few non-French speakers participate in debates in France and read French writings on humanitarian action plays a major role in creating this perception. This article summarises the key critiques, and reflects on the debate.

Criticisms of Sphere

 The first set of critiques concerns the minimum standards. It is argued that Sphere does not take into account temporal, contextual and socio-cultural differences: achieving ‘universal’ minimum standards might not be possible in the initial phase of an emergency response, or in volatile contexts with, for example, ongoing displacement or problems of access and security. Moreover, although Sphere argues for close consultation with, and the involvement of, target groups, it also seems to impose sometimes very detailed standards, which may in fact differ from the wishes of the beneficiaries. 

The second set of critiques concerns the Sphere charter. Since it was developed by NGOs, the argument goes, it has no basis in international law. Moreover, it implicitly sets a whole development agenda: the living conditions of half of the population of Calcutta, for example, would not meet the minimum standards, even in ‘normal’ times. Guaranteeing people’s basic material rights and the right to life with dignity, as proposed in the charter, cannot be realised simply by material assistance. It is intimately linked with social, economic and political rights, which the charter does not address sufficiently.

The third set of critiques concerns the appropriateness of Sphere as a tool for quality assurance. Do its standards and indicators direct attention to the most important issues? Is it not more important to review crisis responses in terms of the broad political and strategic choices that are being made, for example who decides which areas are given priority for demining in a national demining programme? Is it not the impact of an action that is most important, rather than whether technical standards have been met? Does one measure, for example, the quality of a prosthesis only in technical terms, or in terms of whether a person can afford it, walk with it, and repair and replace it locally? Should evaluations not rely more on participatory methods to gauge beneficiary satisfaction, rather than checking whether a project output met the standards?

The fourth set of critiques concerns the possible abuse of the Sphere standards – the ‘Fear project’. Sphere, it is argued, is unduly presented, or perceived, as the major management and evaluation tool for disaster response. The emphasis on technical standards and indicators and on service delivery can have profoundly negative consequences. Sphere’s ‘technocratic’ approach to ‘professionalism’ reduces the importance of global solidarity between peoples, and the importance of individual commitment (the ‘volunteer spirit’). The concern to meet the standards can also stop NGOs from being innovative and taking risks so as to avoid uncertain outcomes. Finally, signing up to codes and standards is becoming a cheap way of obtaining an image of professionalism and getting ‘accreditation’. But signing says nothing about competence and performance in the real world, where agencies may continue with business as usual. Simultaneously, there is the risk of abuse by institutional donors, which may conceivably fund only those agencies that have signed up, or that are meeting the standards. This works against smaller and younger associations, and against many southern and eastern NGOs.

 The final critique concerns that of the duty-bearers. That Sphere and the Ombudsman project are NGO initiatives whose primary target audience seems to be NGOs gives the impression that it is NGOs which also have the primary duty or responsibility with respect to people’s rights. But as the situation in Iraq under sanctions shows, NGOs do not have the legal and political clout nor the material capacity to bear this responsibility. The real duty-bearers are governments. Sphere keeps governments and donors out of the picture, and distracts from the politics of humanitarian action and the international legal obligations that governments have entered into.

Criticisms of the Ombudsman project

The former Ombudsman project comes under fire because of its perceived association with Sphere and the 1994 Code of Conduct for the Red Cross movement and international NGOs. It is rejected primarily because it is seen as a mechanism to address one major weakness in the Sphere project and the code: the question of who ensures that principles and standards are adhered to, and who enforces quality performance. Such a policing role would mean an Ombudsman evaluating performance and obliging agencies to apologise, and possibly even pay compensation, where their poor practices have caused harm, and perhaps even assuming the authority to exclude poor performers from field operations. This is deemed to be unacceptable, and a violation of the independence of NGOs. Moreover, like Sphere, an Ombudsman office is held to have no basis in international law, and is perceived as turning an undue spotlight on NGOs as duty-bearers. It is a false solution to the problem of quality assurance. Quality assurance, it is argued, has to come through stronger evaluation, learning, coordination and advocacy.

The debate in perspective 

The argument is far from concluded. But it is time for all sides to start putting things into perspective and engaging in a more constructive debate.

The legal basis

There are three points to make here. First, the Sphere charter and codes of conduct cannot substitute for, and should not distract from, existing international laws, and the obligations that states have entered into by signing and ratifying them. Second, not everyone agrees that there is no basis in existing law for a humanitarian charter or an accountability mechanism. Third, the fact that NGOs collectively propose a certain interpretation of rights cannot by itself be objectionable. Laws are open to evolution. Many NGO coalitions advocate for more stringently protective interpretations of existing legal instruments, while others have been the source of new international treaties, such as the Ottawa convention on landmines. In its latest World Disasters Report, the Federation of Red Cross and Red Crescent Societies has called for a body of law covering international disaster response. The circus in Kosovo, where the ‘international community’ is arguing over institutional self-interests while at the same time telling the Kosovars to live harmoniously together, shows all too well how misplaced many agencies’ centre of gravity has become.

The duty-bearers

This is a crucial issue. It is important to keep the spotlight on the responsibilities of governments and armed groups controlling populations. The array of national and international actors in the humanitarian response ‘system’ leads to diffuse and sometimes confused responsibilities, and the extent to which international actors should and can temporarily relieve a given state or armed group of its duties is not clear. Still, in the confusion it is possible to distinguish between actions and decisions that are clearly within the control of one actor or another, and others that clearly are not.

Abuse by donors 

Staff in donor administrations (including nongovernmental fundraising platforms such as the Disasters Emergency Committee in the UK) are intelligent people. Nonetheless, there is a risk that an administrative short-cut to quality control will lead donors to rely on agencies’ signing up to codes and standards, instead of properly monitoring and evaluating their performance. There are precedents, such as the reduction of the Local Capacities for Peace project to ‘Do No Harm’, and the use of this soundbite to deny NGOs funding in politically sensitive situations. Additionally, the terms of reference of evaluations as a matter of ‘standard’ good practice should include the performance of the funding agency and its impact on the performance of the operational agency. Finally, accountability in the light of a basic right to life with dignity would also call into question the increasing influence of geostrategic considerations in international aid flows. But the fact that something can be abused is not enough by itself to deny it existence. Everything can be abused, including French wine and freedom of speech. We try to control the abuse, not ban the item or the right.

Abuse by NGOs

 There is ample evidence that NGOs signing up to these codes have continued to operate as before, making it a token gesture. Field-level codes in, for example, Liberia or Sierra Leone were disregarded as soon as the outside pressure on agencies eased. Many agency staff still do not know about the 1994 Code of Conduct, or have no guidance on what it means in practice; others are no longer happy with the code as originally formulated. The 1996 InterAction field-cooperation protocol had no noticeable impact on practice. It can be argued that nothing is more damaging to the reputations of NGOs than their inability to take their own commitments more seriously. The People-in-Aid code and the Sphere project may have more influence and impact, because a project was built around them to support agencies working with them. At the same time, a bland rejection of attempts to articulate external benchmarks against which performance can be reviewed is equally objectionable: it does not weed out poor performance and, while it may preserve the ‘voluntary spirit’, this can lead to amateurism. It can also become a way of evading critical examination and accountability.

Good practice 

Humanitarian action usually involves managing dilemmas. There are no simple best solutions, and good practice to a large degree may amount to making the best possible choice under difficult circumstances. The Sphere handbook highlights standards and indicators, while paying comparatively little attention to context and scenario analysis, and to the processes and arguments that inform decisions. The RRN/HPN Good Practice Reviews, by contrast, highlight arguments and contextual considerations, and leave it more to the user to develop the action most appropriate to the context. Nonetheless, it is an exaggeration to argue that the Sphere handbook does not challenge users to think through a programme’s design and relevance. In the end, it is the impact that counts, but the nature of the decisions and the constraints on them need to be taken into account.

Rule or tool 

An especial objection stems from the perception that the Sphere standards intend to be a universal onesize-fits-all solution, or an immutable rule. This interpretation is partially fed by the connotations of the word ‘standard’ – ‘norme’ in French – but also by Sphere’s own insistence that the minimum standards are a universal right. At the same time, the Sphere handbook is probably used more as a tool, a benchmark rather than a straightjacket. The question then becomes whether aspiring to meet minimum standards or their progressive realisation is good enough, or just another excuse.

Unhelpful positions 

Sphere’s value depends very much on how intelligently it is used. The hype around it, both in favour and against, has been exaggerated. Agencies were doing quality work for years before the Sphere project, and will continue to use references other than Sphere to do so. The Sphere handbook by no means encompasses all the aspects of humanitarian action in which quality work is required. That does not invalidate it, but it does put it in perspective. On the other hand, the Sphere project has created an unprecedented degree of reflection and debate about quality work, rights and accountability, which in itself is a major achievement. And there is growing testimony of how that reflection is filtering down to field level.

Another mistake is the appeal to ‘size’, and the implicit claim of consensus. Is the fact that many people contributed and that many agencies actively work with the Sphere handbook decisive? No. It should be remembered that the MSF movement originated during the Biafra war out of a breach of the then accepted ‘code of conduct’. The origins of Handicap International lie in a rejection of then-accepted standards of treatment for mine victims in Cambodia. There are also examples from outside of the French NGO world of agencies pushing boundaries, and even trespassing beyond them.

The argument that Sphere and the Code of Conduct would be the key references for a humanitarian ombudsman, and ascribing to this function a policing role, is also misplaced. First, it elevates interagency references above the opinions and priorities of the target populations, and above locally developed and agreed references. Second, there is significant, though largely concealed, unease about some of the principles in the code, and the wide room for interpretation in their application. There are other possible references, as well as significant experience with increasing transparency and accountability to target groups.

Finally, it is clear that an ombudsman-type function must operate at different levels, ranging from the district to the international ‘humanitarian capitals’. No one entity will be able to do this. At the same time, field research has indicated widespread agreement on the need for an independent function to strengthen the voice of target populations, and to improve the listening skills and responses of those claiming to bring benefits to others.

Those who are critical of Sphere and the Ombudsman project have also made mistakes. The biggest has been to turn the debate into a French–Anglo-Saxon argument. First, there are other thinking races on the globe beyond just these two. Second, there are French humanitarians who are prepared to engage with Sphere and the Ombudsman/Accountability project, while many of the concerns raised by the ‘French’ are shared by, and debated among, other nationalities. Third, French chauvinism goes against the spirit of universality that should be one of the driving values of humanitarian action, and introduces a dark nationalism that these same agencies see as a major contributing factor to the conflicts which cause humanitarian needs. This is deplorable, and it is good to see it causing growing unease within several French NGOs.

Misrepresentation is frequent, even in writing. ‘Anglo- Saxon’ positions and practices are bluntly labeled ‘technocratic’ and ‘instrumentalist’, reducing target populations to mere ‘consumers’. It is more than hinted at that Anglo-Saxon humanitarian action is heavily influenced by business concepts. Such a portrayal reflects more an undertone of anti- Americanism than the reality of what most so-called ‘Anglo-Saxon’ aid organisations actually do. Critical – and sometimes outright hostile – readings ignore the nuances and reservations prevalent in the respective project documents.

The corollary of this is uncritical praise for French NGO action. The ‘advice on the Ombudsman project’, from the National Consultative Commission for Human Rights to the French government, refers to various French institutions and platforms for evaluation, coordination, training and learning, and argues that advocacy is a better way of improving quality and accountability. It also claims, mistakenly, that France, with the Groupe URD, is the only country in the world with an interagency platform that combines research, training, evaluation, learning and advocacy. Painting the humanitarian world in black and white in this way deceives the French public and the French authorities, and makes it difficult for outsiders to listen sympathetically to what are sometimes valid points. The irony is that an evaluation of the response of French NGOs to Hurricane Mitch, conducted by the same Groupe URD, shows that French NGOs and evaluators are not vastly superior to their ‘Anglo-Saxon’ and other colleagues, but are struggling with exactly the same issues, and exhibit often very similar weaknesses. 

In the heat of the debate, the mostly francophone opponents of Sphere and the former Ombudsman project are missing many points, and risk throwing the baby out with the bath water. First, they underestimate the influence their valid concerns have already had, and which is visible in the rethink of the Ombudsman which changed it into a humanitarian accountability project. Second, they foreclose constructive involvement with the piloting of these projects, an undertaking in which the French have a strong tradition. Third, the elevation of the principle of NGO independence to the status of a sacred cow, whereby every external critical look at NGO performance is treated as a violation, risks becoming seen as an excuse to evade accountability. NGOs have been in the forefront of the battle to make governments, civil administrations and corporations adopt codes of conduct and articulate standards against which they could be held more accountable. It would be an ugly inconsistency, and highly undermining of their credibility, for them to refuse any benchmarks against which they could be held accountable.

The discussion about the nature of the benchmarks and how aid agencies should be held accountable is not closed. But the real problems of interagency competition, poor performance and weak or nonexistent accountability need to be addressed. The politics of humanitarian action need watching and require critical engagement, but this does not mean that the quality of performance cannot be inquired into at the same time.

The current acrimony and polarisation are unhelpful and damaging. There is a need not only to take valid concerns and criticism seriously, but also to end hostile readings, misrepresentations and chauvinism. Both the Sphere project and the Humanitarian Accountability Project are now to be piloted. This should be done not only in a spirit of true action-research, with constant critical reflection and adaptation, but also with the will and persistence to find workable and acceptable ways of enhancing the quality of performance and agency accountability. A ‘project on quality’ can be as valid a vehicle for this as any other. Regardless of nationality, everyone’s experience, commitment and expertise need to be drawn upon.

Resources

• ‘On the Job Training on Sphere’, International Health Exchange, August 2000, pp. 13–16.
• ‘Can We Measure the Quality of Aid?’, Humanitarian Affairs Review, no. 9, 2000, pp. 24–30. .
• Chaloka Beyani, The Legal Framework for an International Humanitarian Ombudsman, <www.oneworld.org/ombudsman/articles/legalfin.html>, October 1999.
• J.-L. Bodin, ‘Vers une Derive de l’Humanitaire?’, in Geopolitique de la Faim (Paris: Presses Universitaires de France/Action contre la Faim, 2000), pp. 253–56. .
• Ian Christoplos, ‘Humanitarianism, Pluralism and Ombudsmen: Do the Pieces Fit?’, Disasters, vol. 23, no. 2, June 1999, pp. 115–24. .
• Commission Nationale Consultative des Droits de l’Homme, Avis sur un Projet d’Institution d’un Mediateur Humanitaire, Paris, 2000. .
• ETIKUMA 99: Colloque Européen de l’Ethique Humanitaire, Les Codes de Conduite: Référence Ethique et Gage d’Efficacite pour les Actions Humanitaires Internationales du IIIe Millénaire? (Lyon: Bioforce, 2000).
• Lola Gostelow, ‘The Sphere Project: The Implications of Making Humanitarian Principles and Codes Work’, Disasters, vol. 23, no. 4, December 1999, pp. 316–25.
• Groupe URD, ‘Les Dangers et Incoherences des Approches Normatives pour l’Aide Humanitaire’, <www.urd.org/rech/sphere/dangers.htm>, 1999. .
• Groupe URD, ‘Plate-forme des ONG pour une Autre Approche-Qualité de l’Action Humanitaire’, <www.urd.org/rech/sphere/platform.htm>, 2000. .
• François Grunewald, La Sphere en Question, mimeo, Groupe URD, 1998.
• International Federation of Red Cross and Red Crescent Societies, ‘The Sphere Project: Everybody’s Business’, in World Disasters Report 1999 (Geneva: IFRC, 1999), pp. 115–30. .
• Simon Maxwell, What Can We Do with a Rights-Based Approach to Development?, ODI Briefing Paper No. 3 (London: Overseas Development Institute, 1999), <www.odi.org.uk/briefing/3_99.html>, 1999. .
• P. Perrin, ‘Standards et Assurance de Qualité’, in Geopolitique de la Faim, pp. 247–51. .
• The People-in-Aid project: <www.peopleinaid.com>.
• D. Puillet-Breton, ‘Une Code de “Bonne” Conduite?’, in ibid., pp. 231–45. .
• Peter Raynard, Mapping Accountability in Humanitarian Assistance, <www.odi.org.uk/alnap/pdfs/praccountability.pdf>, Active Learning Network for Accountability and Performance in Humanitarian Assistance (ALNAP), May 2000. .
• Hugo Slim and I. McConnan, A Swiss Prince, a Glass Slipper and the Feet of 15 British Aid Agencies (London: Disasters Emergency Committee, 1998). .
• N. Stockton, ‘The Search for Standards and Accountability in Emergency Relief Operations’, speaking notes at the Institute of Development Studies (IDS), University of Sussex, <www.ids.ac.uk/ids>, 2000.

The author can be reached via Oxfam GB