The Tsunami Evaluation Coalition: implications for practice

This article seeks to do two things. First, it summarises the recommendations of the Synthesis Report of the joint evaluation of the international response to the tsunami, carried out under the auspices of the Tsunami Evaluation Coalition (TEC). Second, it considers some of the possible implications of these recommendations for practice. While the tsunami was a sudden-onset natural disaster and the worst affected countries were middle-income countries with relatively well-developed local capacities, many of the operational problems encountered in the tsunami response mirror those seen not only in other responses to natural disasters, such as Hurricane Mitch in Central America in 1998, but also in responses to complex political emergencies like Rwanda in 1994 and Kosovo in 1999. Thus, while some of the lessons learned are particular to the tsunami response, the Synthesis Report reflects themes and trends that have been evident in many other emergencies. TEC recommendations are therefore more broadly applicable, and include strategies and systems to improve humanitarian action as a whole.

Summary of TEC findings

The TEC studies find that the international response to the tsunami disaster helped the affected people and reduced their suffering. They identify many examples of good practice in emergency response, and some welcome innovations. However, overall the studies conclude that the response did not achieve the potential offered by the generous funding available.

The TEC studies also find that:

• Local people provided almost all immediate life-saving action and early emergency support, as is commonly the case in disasters. Furthermore, international agencies experienced major problems in scaling up their own response.
• International action was most effective when enabling, facilitating and supporting local actors. International agencies often brushed local capacities aside, even though they subscribe to norms and standards that call for engagement with and accountability to local actors.
• There were many examples of poor-quality work, not only in the relief phase (largely from inexperienced agencies) but also in the recovery phase. Several initiatives have been launched to improve the quality of humanitarian work, but none has an effective mechanism to sanction agencies for failing to meet their provisions.
• The tsunami highlighted the arbitrary nature of the current funding system for humanitarian emergencies. This system produces an uneven and inequitable flow of funds for emergencies that encourages neither investment in capacity nor responses that are proportionate to need. Despite some donors’ commitment to the principles of Good Humanitarian Donorship (GHD), donors often took funding decisions based on political calculation and media pressure.

Recommendations: summary and possible implications

The TEC Synthesis Report makes four recommendations: around ownership (who has decision-making power and control over a response), capacity, quality and funding. These are all about one central idea: that the humanitarian aid community relinquish ownership to affected populations so that they may direct their own relief and recovery strategies. This change needs to be supported by more equitable and proportionate funding, the development of disaster response capacities, a greater focus on risk reduction and a system for controlling the quality of work done by humanitarian agencies.

Ownership and accountability

Synthesis Recommendation 1: The international humanitarian community needs a fundamental re-orientation from supplying aid to supporting and facilitating communities’ own relief and recovery priorities.

The TEC studies go beyond the usual call for greater consultation and suggest that the affected population itself sets the priorities and draws up the plans for emergency response and recovery. While many agencies claim to put affected people’s priorities first, the tsunami response clearly showed that, at least in the initial stages (the Synthesis Report covers the first 11 months of the response), this did not happen to any meaningful degree. The reason for promoting ownership is simple: the affected population is far more knowledgeable about their context, needs and capacities than external agencies. Interventions that are in tune with people’s priorities are therefore more likely to make the best use of their capacities, resulting in a more appropriate, effective and efficient intervention.

The Synthesis Report authors believe that accountability is closely linked to ownership. Those implementing the response should account for their actions, first and foremost to the affected population. Agency transparency is not in itself enough to ensure accountability. The affected population must have the capacity to analyse information provided to them by intervening agencies, and mechanisms to ask questions and to influence and control policies and actions.

Change on this scale is not going to be easy. Nevertheless, the potential advantages in terms of more effective, appropriate and sustainable aid are convincing. While some still see humanitarian action as concerned with relieving the distress of the affected population without addressing underlying causes, Article 9 of the Red Cross/NGO Code of Conduct states that ‘relief aid must strive to reduce future vulnerabilities to disaster as well as meeting basic needs’.

Capacity

Synthesis Recommendation 2: All actors should strive to increase their disaster response capacities and to improve the linkages and coherence between themselves and other actors in the international disaster response system, including those from affected countries.

The tsunami response clearly showed that prior linkages between different partners improved the chances for quality programming. This has been recognised in the UN’s Delivering as One: Report of the Secretary-General’s High Level Panel, which acknowledges the TEC for highlighting the need to ‘enhance local preparedness [and] reduce long-term vulnerability’. The existence of institutions like local Red Cross branches and local NGOs also helped to save lives, while those countries with proven national disaster response procedures, such as India, were able to respond far more quickly.

At the international level, the Synthesis Report suggests that one area that needs considerable focus is agency response capacity. Staffing issues, both in terms of the lack of adequately qualified staff and high levels of staff turnover, are one of the biggest constraints on agency capacity. Other emergency responders, such as the military, also need to develop their emergency response capacity, and their linkages to the humanitarian response community. Some suggest that there needs to be greater engagement between civil and military actors to prepare for emergencies.

The practical implications of this recommendation are broad:

• Agencies should support the development of national disaster response capacity in line with the Hyogo Framework for Action.
• Agencies should support community-based disaster reduction.
• Agencies need to consider ways of increasing their own emergency response capacity.
• Agencies should develop linkages with other potential humanitarian actors.

This can only happen with donor support. A good start is the UK’s Department for International Development (DFID) policy to use one-tenth of its natural disasters budget for disaster preparedness and mitigation. DFID’s approach is unusual, however, and agencies may have to engage in advocacy with donors on this issue.

The issue of linkages and local capacity is also critical. It is only through regular contact that different agencies can build up a picture of the capacities of potential partners in an emergency response, and set up the networks essential for a joined-up response.

Another key aspect is strengthening community disaster response capacity through, for example, risk reduction work. This is integral to the ‘build back better’ approach. It is thought that strengthening community capacity in this way will put the community in a much better position to respond to, and manage, any future disaster response.

Quality

Synthesis Recommendation 2: The international relief system should establish an accreditation and certification system to distinguish agencies that work to a professional standard in a particular sector from the others.

Despite the huge amounts of funding made available, the tsunami response suffered from the common problems of inappropriate aid, lack of consultation with beneficiaries and competition between agencies. While the humanitarian sector has high standards, there are no rewards for agencies that meet them, or penalties for agencies that do not. The lack of accurate information to the donor public on the performance of agencies makes this problem worse, and the media cannot be relied upon to provide the sort of information that would allow donor publics to assess performance.

Accreditation and certification

TEC reports echo the Joint Evaluation of Emergency Assistance to Rwanda in calling for certification. One key purpose would be to improve information flows by measures such as the compulsory publication of independent external evaluations and independent surveys of beneficiary opinion. Providing good information to the donor public and taxpayers will provide external pressure for improving the quality of humanitarian response.

Introducing a certification and accreditation system for international humanitarian aid actors would have significant implications for both existing and new actors. Certification and accreditation have a real cost, and this is only justified if it leads to better quality. This can only happen if the conditions set for certification and accreditation:

• encourage greater agency transparency and accountability to beneficiaries, through publishing evaluations and internal quality control reports;
• encourage agencies to concentrate their efforts, and develop deeper competence in specific sectors;
• encourage agencies to invest in their own emergency response capacity;
• discourage new agencies from entering the sector unless they have an obvious competence; and
• encourage collaborative, rather than competitive, relations between agencies.

Clearly, such a process is going to take time to negotiate and set up. However, if agencies fail to act on this issue governments could impose certification systems that bring increased costs without any matching benefits for affected populations.

Funding

Synthesis Recommendation 4: All actors need to make the current funding system impartial, more efficient, flexible and transparent, and better aligned with principles of good donorship.

Funding for humanitarian aid has grown over the last two decades, both in absolute terms and as a percentage of all overseas development aid. However, the growth is uneven. There is also great variation between and within countries. In principle, funding for humanitarian action should be proportionate to need; in practice, however, it is not. The total funding for the nearly two million people affected by the tsunami was $13.5 billion, or over $7,000 per person. This compares with $3 per person for the 36 million affected by floods in Bangladesh in 2004. The Synthesis Report considers that this does not mean there was too much funding for the tsunami, but rather that most emergencies are funded far inferior to need.

The major initiative on principled donorship is the GHD. However, the tsunami response showed many deviations from the GHD principles. These included funding decisions based on political and media pressure rather than needs assessments, and a general lack of beneficiary involvement in design, implementation, monitoring and evaluation. That said, the GHD initiative is still relatively young, and does represent a good chance for a change to a more principled funding system. Clearly, independent external review, added to the existing peer review by the Development Assistance Committee (DAC), could improve compliance with GHD. Another initiative has been the reform of the UN Central Emergency Response Fund (CERF), which reserves one-fifth of its funding for ‘forgotten emergencies’.

Thus, official funding for humanitarian action may be moving towards a more just system. However, the same is not true of funds raised from the public. This will only become more balanced when both the public and the media have a greater understanding of the nature of global humanitarian needs and the complexities of humanitarian interventions.

One key part of humanitarian funding reform is the proportion of funds that agencies devote to developing disaster response capacity, both within their own agencies and within communities and countries at high risk of disasters.

Pulling it all together

All four TEC Synthesis Report recommendations are closely intertwined and support one another. The question is, what now? How should the humanitarian community take the TEC findings and recommendations forward?

The picture so far

Clearly, responsibility lies with all those involved in the TEC. The TEC represents the largest study of a humanitarian response since the Joint Evaluation of the Rwanda response in 1996. Over 40 agencies were involved in one or more of the five thematic evaluations and/or the Synthesis Report, and over 65 agencies kept themselves actively informed about the process.

Currently, the humanitarian sector is operating in what might be termed a ‘post-tsunami policy climate’, and this climate has been influenced by the TEC. In evaluation utilisation terms, the TEC is therefore having ‘conceptual’ and ‘legitimising’ use. For example, the first two Synthesis recommendations have been recognised as ‘particularly relevant to WFP’ and have added ‘stimulus to some of the initiatives currently being undertaken’; in the Netherlands, ‘The recommendation that donor governments need to aim for neutral, flexible and transparent financing mechanisms supports the current policy that intends to make optimum use of pooled funding/CERF-like mechanisms’. TEC recommendations have also found their way into sector-wide initiatives such as the NGO Impact Initiative, and evidence is mounting of direct or ‘instrumental’ use, in Danish NGOs’ work on certification, for example, Cordaid’s efforts to improve its needs assessments and beneficiary accountability processes, and ALNAP’s incorporation of joint evaluation into its workplan.

Next steps

In arguing for a fundamental, integrated re-orientation, the TEC is setting a new challenge – not only to the humanitarian sector, but also to those involved in development and recovery. This challenge will require a greater level of partnership and cooperation between humanitarian and development agencies, and between these actors and affected populations, national and local governments, new private sector responders and the military.

It will be up to senior policy-makers and programme managers to continue to look at the TEC’s recommendations and factor them into their future strategies and programming approaches, where appropriate. It will also be important to continue to examine TEC recommendations in inter-agency networks and initiatives in order to maintain an integrated approach to reform. One suggested first step is included in the NGO Impact Initiative report on accountability. This recommends that agencies undertake an audit of their accountability to affected populations. Such an audit might usefully be done by all agencies, however, and not just NGOs. It could seek to highlight some of the issues that agencies need to address in their own work to begin the overall re-orientation that the TEC studies call for.

However, leadership for sustainable change has to come from the top. Agency boards, trustees or other governing bodies of the Red Cross, NGOs, donors and UN agencies could usefully develop a comprehensive picture of what a fundamental, integrated re-orientation might look like. Ideally, this would happen collaboratively within inter-agency networks – and include affected-country actors. In the interim, agencies could decide which of the TEC recommendations they accept, and set up a process at board level to oversee the agency’s progress towards implementing them. Only by involving governing bodies in this way, and by working at an integrated level, can we bring about the changes needed to improve the quality and efficiency of the humanitarian response system, let alone a much larger fundamental re-orientation.

Rachel Houghton has been involved with the TEC since it was established in February 2005. She has coordinated the Coalition since July 2006, and prior to that was the TEC’s researcher and deputy coordinator. The TEC’s website address, where you can download all the TEC reports, is: www.tsunami-evaluation.org.