An assessment of mortality studies in Darfur, 20042005

War epidemiology has emerged as a powerful tool for expanding our understanding of the impact of war on civilians, and for influencing policy aimed at protecting civilians from systematic harm. Mortality statistics generated from rigorous field studies can raise awareness of increased risk among vulnerable populations, and can serve as evidence that military groups and civilian combatants have violated wartime codes of conduct. Recent mortality studies illustrate the widespread (and often controversial) attention that statistical evidence of civilian casualties can generate. A study of mortality in Kosovo, published in 1999, documented a pattern of targeted assault on older men. An ongoing study in the Democratic Republic of Congo suggests that almost four million civilians, the majority of them women and children, have died in war and its aftermath. In Iraq, a study published last year (and reported in Humanitarian Exchange 29 in March 2005) indicated that upwards of 100,000 civilian deaths may have resulted from the invasion. The results of these careful mortality assessments are not easily ignored, and have sparked serious debate among policy-makers and humanitarian organisations.

The protracted war in Darfur has led to the death and suffering of many inhabitants of the region, and has prompted a major international relief response. The circumstances and consequences of attacks on civilians and civilian villages by armed militia, with the tactical air and ground support of the Sudanese government, are not in substantial dispute. What remains unclear is the cumulative mortality from this conflict. This lack of clarity regarding the human costs of the war in Darfur does not stem from any absence of effort to try to establish the facts. In the last 12 months, four different surveys addressing issues of mortality have been undertaken. These have either been released to the public, or published in the peer-reviewed literature. In addition, three desk analyses have been submitted via the web and e-mail to the policy and humanitarian community. The discussion that follows summarises these studies (methodology, results and limitations), and gives recommendations for how best to approach further attempts at arriving at mortality estimates, and how to think about the mortality dimension of this conflict, in the absence of good overall data from the field.

Field surveys

MSF surveys, West Darfur, April–June 2004

In a report published in the Lancet in October 2004, MSF used a two-stage household-based cluster survey in four IDP sites in West Darfur. Together, these sites had an estimated IDP population of 215,400 people. The survey focus was on mortality experienced by respondents during ‘pre-displacement’ (the period of flight) and ‘post-displacement’ (the period in camps or settlements).

The MSF teams found high crude mortality rates (CMR) for the period of flight, and persistent high rates (though lower than pre-displacement) from violence in the IDP camps after settlement. CMR for pre-displacement populations surveyed in three of these four IDP areas ranged from 5.9 to 9.5 (for populations sampled in El Geneina, the mortality recall period – one month – did not include the period of flight). CMR for post-displacement (camp mortality) for all four areas ranged from 1.2 to 5.6. Deaths from violence were recorded separately from all other causes, and constituted the majority of deaths during the flight period. A finding common to all four areas was a marked loss of males in the 15–49 age group.

Important findings of this study include the mortality differentials between the flight and the settlement period, the high proportion of violence-related deaths during the flight period, and the sex ratio among those who died as well as among the survivors, suggesting that violent deaths of males, as opposed to possible disproportionate death from other causes or out-migration, caused the observed loss in the 15–49 group. The limitations of the study are the sampling frame (portions of West Darfur only) and the absence of a sufficient recall period to establish a full profile of deaths during flight and settlement periods for the large subset in this study from El Geneina.

WHO retrospective mortality survey, August 2004

This study (released in September 2004) aimed to estimate the CMR in the 62 days from 15 June to 15 August 2004, among IDPs present at the time of the survey in August 2004. It used a two-stage cluster sample drawn from lists obtained from WFP and OCHA for ‘accessible’ areas within the three states of Darfur. Deteriorating security in South Darfur meant that the study had to be suspended there. Crude mortality rates were derived from a two-month recall period for IDPs.

Diarrhoea was the main cause of death in all three locations. For all ages, violence or injury accounted for 21% of deaths in North Darfur, 12% in West Darfur and 10% in Kalma camp. A striking finding in all three areas, as with the previous study, was the relative loss of males between 15 and 49 years of age.

Major limitations of this study are the sampling frame, access to those within the sampling frame, and limited recall period. Another issue is the inclusion of interviewers who were in the employ of the Sudanese government.

MSF surveys, South Darfur, August–September 2004

In a report published in JAMA in March 2005, MSF described findings from three surveys conducted in South Darfur in August–September 2004. The surveys were based on two-stage cluster sampling among an estimated 137,000 IDPs in Kass, Kalma and Muhajiria. They reported on crude and under-5 mortality rates and nutritional status. Deaths were recorded for a recall period of 121 days for Kass, and 30 days for Kalma and Muhajiria (this shorter period due to ‘priorities of the operational teams’).

Limitations of this study include the sampling frame and the short recall period. The study reflects the mortality profile associated with IDP settlement, rather than the mortality arising from pre-displacement attacks on villages and the impact of flight.

Centers for Disease Control and Prevention (CDC) emergency nutrition and crude mortality survey, September 2004

This study, conducted in Darfur in September 2004 and released in February 2005, was intended to assess nutritional status among children between six and 59 months old and their mothers. It also aimed to arrive at an estimate of crude mortality in the same population. It was based on a two-stage cluster sample from a list of 1,655,988 people (IDPs and host populations) in 140 locations across Darfur.

In the 880 households (5,470 individuals) selected for the study, the mean length of time in residence in the area was 7.5 months. Malnutrition rates were found to be high (21.8% GAM, 3.9% SAM) with serious rates of micronutrient deficiency. Findings for crude mortality, however, were less striking: based on a seven-month recall (to February 2004), the overall CMR of 0.72 was close to the estimated baseline (0.5) for the period before the onset of the conflict. The female CMR was 0.4 (very low, even in a stable sub-Saharan population), and the under-5 MR was 1.03. These are surprising findings in a population that has undergone conflict-induced forced migration, and with high levels of malnutrition. Of the 81 deaths reported during the recall period, cause of death was not recorded for 39 (48.2%); 13 deaths were noted as due to violence.

Limitations of this study, from a mortality standpoint, were acknowledged in the CDC report. Issues included the heterogeneity of populations across a vast area, sampling frame, sample size, access to those within the sampling frame, recall period, resident time in the area, and recording of the cause of death.

Desk assessments

USAID mortality projections May 2004

In spring 2004, USAID released on its website a graphic projection of death rates in Darfur from disease, violence and malnutrition, based on estimates of baseline malnutrition and mortality, and gaps in food supply from January 2004 to December 2005. This projection resulted in an estimate of 300,000 excess deaths by the end of 2005.

Mortality estimates from Eric Reeves

In autumn 2004, Eric Reeves, a professor at Smith College in Massachusetts, began a running tally of estimated mortality from the onset of the conflict in February 2003. His October 2004 report, distributed over e-mail, estimated total mortality at 300,000 as of that date. This was based on an extrapolation from an August 2004 survey of Darfurian refugees in Chad, conducted by the Washington-based Coalition for International Justice (CIJ). This found that 61% of those interviewed reported witnessing the killing of a family member. Using a figure of 1.6 million IDPs in Darfur, and adding the 200,000 known refugees in Chad, Reeves applied this rate and determined that approximately 200,000 people had been killed by violent means since the onset of the war. He added to that another 80,000–100,000 dead from disease and malnutrition, basing this on monthly estimates from the UN and NGOs.

Subsequent updates have refined estimates for the numbers of people dying from disease and malnutrition. By March 2005, Reeves was estimating that 380,000 people in Darfur had died from disease and violence-related causes since the onset of the war. He also suggested that the monthly death rate from all causes may rise to 15,000, as food supply and security conditions deteriorate in the absence of cohesive political action to bring the conflict to a close.

Mortality estimates from the CIJ, April 2005

In April 2005, the CIJ estimated that almost 400,000 people had died in Darfur since the start of the conflict, from violence, disease, starvation or exposure, with the death rate continuing at about 500 per day, or 15,000 per month. These figures, developed by academics from Northwestern University in Chicago and the University of Toronto, were derived from the ICJ’s August 2004 study and from WHO estimates from September 2004.

Constraints of the context and the limits of survey tools

These studies each have their strengths and weaknesses, but the major flaw in all of the field studies, from the standpoint of mortality surveys, is that the sampling frame did not account for the complex pattern of attack and flight that produced the displacement in Darfur, and did not address the fact that many of the displaced survivors were in areas inaccessible to outside observers. Although accessibility has improved, an unknown large number, in the hundreds of thousands or millions, may well be clustered in isolated pockets of misery, not yet reached by humanitarian actors. An important population of refugees – the 200,000 people inside the Chadian border – was not sampled in any of these studies, so their mortality experience, which is probably distinct from that of people in IDP camps within Darfur, has not been examined.

None of these issues was unknown to the teams that conducted these studies. Constraints on logistics, security and cost imposed daunting barriers. Furthermore, the focus of several of these studies was more on nutrition than mortality, and directed towards assessing camp conditions, rather than arriving at cumulative data on mortality from the conflict. Of the four formal field surveys, only the MSF’s Lancet study in West Darfur explicitly tried to adapt the standard nutritional survey into a retrospective mortality survey that would take into account the temporal and dynamic complexity of the conflict. The results indicate that mortality from conflict-related violence was greatest in the period leading up to and including the events that caused the population to flee. Hence this survey establishes that mortality studies looking only at deaths in camps after the population had arrived might well underestimate the cumulative number of deaths that had occurred.

Other methodological constraints are familiar ones in surveys conducted in difficult conditions with little security for personnel or for the population: small and unrepresentative samples, an inability to compensate for entire missing households, and inaccuracies in determining cause of death. Even with the careful application of sampling techniques to settled populations, the cumulative mortality experience of the populations sampled varied greatly. The prevailing cluster sampling techniques developed for nutritional surveys could not sufficiently compensate for this complex prior experience, particularly if recorded deaths were not partitioned according to pre- and post-displacement. No methods were employed to account for the possibility that, in some households, all members were probably killed; a retrospective household-based survey would miss these deaths completely, and result in an underestimate of reported total deaths. (However, the observed excess deaths among males would suggest that the loss of complete households is less the pattern than deliberate targeting of men.) Determination of cause of death was more or less complete in these studies, but even where death from violent causes was the focus of the inquiry, there is an incomplete discussion of the fact that many of the deaths attributed to malnutrition and disease are themselves conflict-related – meaning that these deaths would not have occurred had the war not driven these people from their homes, villages and means of livelihood.

Other common sources of error or bias (such as whether the population trusted the interviewers on the teams, whether languages other than Arabic would have been useful, whether security or access issues distorted the rigorous application of cluster sampling in some studies) can also be found to differing degrees.

Some of these limitations in the field surveys were addressed at a qualitative level in the desk studies. Efforts were made to build up from small sample sizes to an overall picture of mortality conditions across time, and efforts were made to aggregate deaths from all causes as attributable to the effects of the conflict on a fleeing and then displaced population. The CIJ’s attempt to work backwards, from memories of survivors to a semi-quantitative estimate of deaths at the time of attack, is creative and has been used only rarely in previous attempts to assess conflict-induced mortality. Yet the desk studies can provide only notional policy guidance, in the absence of empirically detailed studies from many different sectors throughout Darfur.

Recommendations

For a conflict of such complexity and duration, there is no methodological shortcut to an estimate of cumulative mortality and causes of death. The entire region has to be considered the sampling frame; the last complete census conducted in 1983 has to be adjusted as the basis for this frame; baseline mortality rates have to be rigorously derived from existing information; and meticulous cluster sampling must be applied wherever possible. The survey instrument must be partitioned to account for experience and mortality before displacement, during flight and after settlement. Careful attention to the age and sex ratios of survivors and of those who died will yield important information regarding the nature and pattern of violence and vulnerability. The design of the instrument and the analysis of the data need to attend to the possibility that all excess deaths (from disease, malnutrition or apparently non-war-related trauma such as road accidents or suicide) could arguably be attributable to the conflict.

In the interim, the international community must make use of the existing information on mortality in Darfur, as reviewed in this article. It is clear that mortality in areas of displacement settlement varies considerably, reflecting variations in camp management, accessibility and conditions. This information is also highly suggestive of the fact that much of the excess mortality that has occurred probably took place during the events that prompted the displacement in the first place; ongoing mortality surveys are not addressing this historical burden. Finally, whether the cumulative deaths to date are in the low or mid-hundreds of thousands, all studies starkly describe a population subject to intolerable conditions of fear, assault and deprivation. This information should be sufficient to goad the international community into redoubled efforts to bring some relief and security to the people in this region.

Jennifer Leaning is professor of international health at the Harvard School of Public Health. She also directs the Program on Humanitarian Crises and Human Rights, based at the François-Xavier Bagnoud Center for Health and Human Rights at the Harvard School of Public Health.

Michael VanRooyen is Chief of the Division of International Health and Humanitarian Programs at Brigham and Women’s Hospital, and Associate Director of the Program on Humanitarian Crises.