SMART: a collaborative approach to determining humanitarian needs

The way in which humanitarian needs are defined and prioritised can mean the difference between life and death for millions of the world’s poorest people. It is, therefore, critical that donors and humanitarian organisations invest effort and resources to ensure that our understanding is as accurate as possible, and programmes are directed to those most in need.

Responses to emergencies require a consistently accurate picture of the scale and nature of the problems people face. Decisions should be informed by that understanding. But until now there has been no system-wide framework for judging the relative severity of situations. This may change with the Standardised Monitoring and Assessment of Relief and Transitions (SMART) initiative.

SMART seeks to improve the assessment and reporting of humanitarian response. Coordinated by the US Agency for International Development (USAID) and the UN Children’s Fund (UNICEF), it includes donors, UN agencies, NGOs and universities. It advocates a multipartner, systematised approach to generating timely and reliable information for decision-making, and for improving the targeting of resources so that they reach those most in need. SMART seeks to provide the following:

• A standardised, simple epidemiological methodology that will enable comparison of need among emergencies, in order to prioritise resource allocations.
• A comprehensive technical support system to build capacity at all levels, and to coordinate resources.

The goal is ambitious but essential: to institutionalise evidence-based policy and operational decision-making, and reporting on humanitarian crises.

The origins of the SMART initiative

USAID, in collaboration with the US Department of State’s Bureau of Population, Refugees and Migration (State/PRM), the Canadian International Development Agency (CIDA) and other partners, initiated SMART at a workshop in Washington in July 2002. The event gave an official name to work that had started many years earlier. Over the previous decade, donors had come under pressure to demonstrate good use of taxpayers’ money, and results-based performance became a priority. USAID was one of the first US government agencies to embrace the Government Performance Results Act (GPRA) of 1993, and began organising itself to demonstrate results. For USAID’s humanitarian areas, two measures of performance were selected: crude mortality (or death) rates (CMR/CDR), and the nutritional status of children under five.

According to the World Health Organisation (WHO), CDR constitute the most critical indicators of a population’s health status, and are the category of data to which donors and relief agencies most readily respond. A CDR not only indicates the current health status of a population, but can also provide a baseline against which the effectiveness of relief programmes can be traced. If the CDR is low, people are not dying from the wide variety of factors that relief aid seeks to address, including epidemics or lack of shelter, food, water and sanitation. The second indicator, the nutritional status of children under five, was selected because it is closely associated with risk of death. Since severely malnourished children die without timely and appropriate interventions, it was thought that integrating mortality analysis with nutrition status data would ensure a more realistic profile of nutritional status.

USAID began using nutritional status to measure the performance of its emergency food aid in 1997. Two years later, in November 1999, the agency funded a pilot test of a methodology that combined CDR data collection with existing nutrition survey protocols in Sudan. The pilot, conducted by World Vision, indicated that this was a feasible approach, and that the burden of adding CDR to existing nutrition survey protocols was minimal. Adding mortality rates to nutrition and other indicators improved programme planning and implementation, and enhanced advocacy efforts, as the mortality data proved useful in triangulating and validating the nutrition data.

State/PRM adopted the two indicators in 2000, and in January 2002 a joint State/PRM and USAID mission to Europe found support for the use of the two selected indicators. Organisations consulted included UN agencies (the World Food Programme (WFP), the Food and Agriculture Organisation (FAO) and WHO), the Red Cross (IFRC and ICRC), the European Commission, NGOs (Médecins Sans Frontières and Médecins du Monde), academics (the Centre for Research on the Epidemiology of Disasters (CRED)) and the Sphere Project. These discussions resulted in the landmark workshop in July 2002.

At the workshop, WHO presented a discussion note on the use of mortality and nutrition indicators to assess organisational performance. The note, written in response to a draft proposal by USAID, discussed the views of WHO’s various technical departments. The workshop concluded that mortality rates and nutritional status ‘are considered the most vital, basic public health indicators of the severity of a humanitarian crisis’. Consensus was reached on developing a generic, standardised methodology to be used in all emergencies for assessing nutritional status, using current best-practice survey methods as a base. It was agreed that the current indicator, CMR, should not be changed until research findings had shown that under-5 mortality rates were a better alternative. It was also agreed that the standard nutrition status indices to be used are wasting (thinness or marasmus) and edema (kwashiorkor). Wasting is to be measured using weight-for-height ratios. Trend analysis, rather than absolute thresholds, was recommended for determining whether a situation warranted intervention.

There is now general acknowledgement of the usefulness of mortality and malnutrition data for overall humanitarian needs assessment and decision-making. Performance measurement experts believe that, if mortality rates and nutritional status are improving, the humanitarian response system is by and large working. These data provide a means for measuring effectiveness, outcome and performance.

Actionable activities

To translate what is a complex concept, involving many agencies, into tangible actions, SMART developed actionable activities that could be managed independently by various partners as part of the whole. Each component is critical, but in itself limited. The collaborative approach brings all the components together to form a comprehensive support system.

Develop standardised SMART methodology

The first priority was to develop a generic methodology, including a survey protocol and an analytical software programme for standardised data collection, analysis and reporting. UNICEF/USAID coordinated this work, which was carried out by a team of technical experts in various sectors, drawn from several organisations.

‘SMART Methodology Version 1’ was developed with funding from CIDA after a two-year process of consultation and pilot-testing in Chad and Somalia. The SMART methodology is iterative, with Version 1 being the most basic. It will be upgraded as research findings validate hypotheses for alternative methodologies. The methodology balances simplicity (for ease of use in acute emergencies) with technical soundness. It should be applicable by NGOs with some technical support. The draft Version 1 was presented to the public at a meeting hosted by UNICEF Executive Director Ann M. Veneman on 23 June 2005.

Establish a complex emergencies database

The Complex-Emergencies Database (CE-DAT) was established in November 2003 by the Centre for Research on the Epidemiology of Disasters in Brussels, with funding from State/PRM. The online, publicly accessible database is a compilation of quantitative and qualitative information from numerous sources, linked to existing conflict databases. It provides data on the mortality, morbidity and nutritional status of conflict-affected populations. In coordination with the Nutrition Information in Crisis Situations (NICS) system of the UN Standing Committee on Nutrition (SCN), which compiles and analyses nutrition data, CE-DAT serves as a data source for trend analysis and policy recommendations.

Develop comprehensive technical support

Once the SMART methodology is finalised, comprehensive technical support will build capacity at all levels, in particular at the country level. The SMART methodology will be tailored to meet training needs: it will be incorporated into existing graduate programmes, such as the Master’s Degree in Public Health, offered as a short-term certification course, or as a distance-learning course. Initially pilot-tested by Tulane University in New Orleans, a French-language version will be developed in conjunction with interested African universities. The idea behind a capacity-building system is that it ensures that learning opportunities for SMART methods are widely available and incorporated into on-going programmes. This component is critical for institutionalising the SMART methodology and approach into mainstream humanitarian response.

Comprehensive technical support will include a virtual library on health, nutrition, food security and related topics, as well as a listserv and a rapid response system maintained by a technical expert group to guide and review surveys, and respond to enquiries from implementing partners.

Establish an operational research agenda

Ongoing research is needed to further advance the methodology and integrate emerging issues. Illustrative activities include using geospatial mapping to determine population denominators and malnutrition prevalence, and the use of alternative methods for determining mortality rates.

Lessons learned

SMART is addressing longstanding issues important to the international community. It builds on the work of the Sphere Project, which seeks to enhance the accountability of the humanitarian system. It was recognised early on that it would be unrealistic to expect a unanimous commitment among the many organisations involved. That said, the SMART experience has reinforced the notion that it takes only a small critical mass of individuals to achieve major advances. The core vision was held together by a group of talented and credible technical experts, who produced a tool that has received excellent feedback from field practitioners, especially for its ease of use – an important measure of success for the SMART methodology. SMART has also benefited from USAID’s internal expertise, and its years of experimentation with results-based performance measurement. The lessons USAID has learned, and which were applied to the SMART approach, include the need to simplify data collection, ensure ease of use and technical soundness, and aggregate results in such a way that they tell a cohesive story that supports funding.

The involvement of donor policy advisors helped to raise awareness of SMART beyond the technical community; SMART was included in the G-8 Action Plan on Famine, and was endorsed at the G-8 Summit in 2004. Senior decision-makers are now attuned to the importance of mortality and nutrition data in crisis situations. It is, therefore, important to improve our understanding of how to apply these data in decision-making, and how to ensure that the information is reliable. The UN Millennium Project has recommended support for SMART to build and strengthen national and local early-warning systems. Countries suffering from chronic food insecurity, such as Ethiopia, should benefit from the multipartner SMART approach.

SMART is contributing to an ongoing effort to enhance how data is used to inform decisions and measure progress in the humanitarian sector. It provides a practical tool that improves understanding of real need. Key indicators, such as mortality and nutritional status, will bolster efforts by humanitarian organisations to define need and measure the impact of humanitarian assistance. A new initiative, led by the UK Department for International Development (DFID), the UN Office for the Coordination of Humanitarian Affairs (OCHA) and WHO, is building on the SMART indicators and identifying others that could assess the collective performance of humanitarian actors. If this work is successful, it should lead to fundamental reform of the humanitarian system.