Measuring the socio-economic impact of post-disaster shelter: experiences from two Red Cross programmes

Post-disaster shelter programmes aim to meet urgent and immediate housing needs. Although evaluations have highlighted short-term benefits and have helped to improve programme design and shelter options, little attention has been paid to the longer-term socio-economic impact of these interventions. Following an initiative of the Netherlands Red Cross (NLRC) and the International Federation of Red Cross and Red Crescent Societies (IFRC), two long-term studies on post-disaster shelter programmes were conducted in collaboration with Eindhoven University of Technology (TU/e) in the Netherlands. The first considers the long-term socio-economic impact of a shelter programme implemented in 1999–2001 in Vietnam. The second covers a Transitional Shelter (TS) programme implemented by the IFRC and others in Aceh, Indonesia.

This is a standardised methodology for measuring long-term impact, making comparisons between different post-disaster housing programmes more accurate. It is our belief that such a standardised research methodology can give valuable insights into shelter impacts, helping organisations to formulate sustainable shelter responses which consider both immediate and long-term needs.

Research approach

Post-disaster housing programmes are very context-specific. Housing design and implementation must take into account key economic, social, political, environmental and cultural factors, including how households make their living. Thus, analysing the impact of such programmes requires a flexible and context-specific approach. This need is typically answered by embracing a qualitative approach, such as in-depth household interviews and group discussions. Although we acknowledge the value of qualitative methods, the case studies below are analysed in the belief that the long-term impact of housing can be better understood when combining qualitative approaches with quantitative tools of analysis, using a standardised research approach. By agreeing on and using a standardised research approach and indicators, it is possible to determine the impact and effectiveness of different shelter programmes.

Both studies discussed here are based on a research approach which includes a focus on social as well as technical factors, a comparison between beneficiaries of the shelter programme and non-beneficiaries, qualitative and quantitative data, objective and subjective indicators and a focus on socio-economic impact over time. For the latter it is best to analyse housing conditions, and the socio-economic and demographic characteristics of households at different points in time (e.g. before/just after project implementation and several years after project termination). In the Vietnam case study, it was necessary to reconstruct baseline data for households in 2000. For the study in Aceh, baseline data was collected in 2008, soon after the project ended; the intention is that, four or five years from now, new data will be collected and compared with the baseline information.

‘Starter houses’ in Vietnam

In 1999, tropical storms and floods struck the central provinces of Vietnam, destroying 55,000 houses and badly damaging hundreds of thousands more. In collaboration with other Red Cross societies, the Vietnamese Red Cross developed a storm-resistant starter-house, consisting of a galvanised steel frame and metal roofing system, positioned on a steel-reinforced concrete foundation. Beneficiary households were responsible for finishing and expanding their houses. In subsequent years, for example, beneficiaries could replace the bamboo mat walls used immediately after the disaster with masonry wall-infill, or even extend the houses. Some 20,000 houses were constructed between 1999 and 2001.

The absence of baseline data was a significant obstacle to determining the socio-economic impact of the shelter programme. Although the initial selection procedures and criteria for identifying beneficiaries were available, wider socio-economic household data was not recorded. In addition, to measure the impact of the shelter programme, it was critical to identify a comparison group which had not received shelter support but had (back in 2000) similar characteristics to the beneficiaries (average age and number of household members, sources of income, level of income, geographical location, etc.). These problems were addressed by analysing secondary data, such as government poverty maps from 2000. In addition, interviews were held with key informants, including community leaders and local Red Cross officials involved in the project in 2000. Semi-randomised sample procedures were applied to select beneficiary and non-beneficiary households. Finally, 94 beneficiaries and 159 non-beneficiaries were interviewed. Household interviews covered objective data, such as housing conditions, but also recall questions regarding changes in living standards since 2000. This research approach – in particular having a comparison group and applying basic statistical analyses – provided valuable insights into socio-economic impact.

Impact of the shelter programme in Vietnam
Overall, beneficiaries were satisfied with the ‘starter house’, especially its good resistance to storm damage. In case study interviews, respondents also noted that house maintenance and repair costs were considerably lower than before the programme. In a few cases, community leaders reported that these costs had dropped to zero. However, it was still not clear whether these maintenance and repair costs were really different for the comparison (non-beneficiary) group, and if so, whether the difference was substantial.

Data analysis indicated that the difference was indeed statistically significant and that the house maintenance and repair costs for beneficiaries were a third less than those of non-beneficiary households. Qualitative interviews indicated that, as a (secondary) effect, beneficiary households were able to spend more money on food, education and health. The quantitative analysis indicated that beneficiaries were more satisfied with their current houses and had more diverse diets than the comparison group. On average, beneficiary households could afford two meat-meals a week, while non-beneficiary households could afford just less than one. Beneficiary households also possessed more consumer goods, such as rice cookers, than non-beneficiaries.

Transitional shelter programme in Aceh, Indonesia

The tsunami of December 2004 displaced over 550,000 people in Aceh. The IFRC and other Red Cross and Red Crescent Societies reacted by implementing a transitional shelter programme.

From August 2005 to December 2007, almost 20,000 transitional shelters were built, targeting people still living in tents or shacks. The objective of the programme was to ‘fill the gap’ between emergency shelter, such as tents, and permanent houses. By providing steel-framed houses with wooden walls, floors and metal sheet roofs, the transitional shelter programme aimed to improve living conditions for a two- to four-year period, giving beneficiaries the opportunity to rebuild their permanent homes and livelihoods. To determine whether the programme did indeed improve socio-economic conditions and thus helped people to rebuild homes and livelihoods, a comparison was made between 300 households that had lived in transitional shelters for a period up to four years, and 300 households that had not. Both groups were already living in permanent houses when the study was conducted, which made it possible to make a good comparison.

The data gathered was used to create a baseline. This sketches the social and economic situation of households now, making it possible to compare the two groups. The data will also be useful for a possible further study in four or five years’ time. When data is collected at two different points in time it is easier to measure actual impact.

Impacts of the transitional shelter program in Aceh
Analysis of the quantitative data shows that the transitional shelter programme has had a strong social impact and a slight economic impact on beneficiaries. The majority of households that have lived in a transitional shelter evaluate the influence of the shelter on their lives as ‘very positive’ or ‘positive’. Furthermore, the baseline data shows that, even after moving to permanent houses, large numbers of beneficiaries still use the transitional shelters either as additional living space or as business premises. This demonstrates that transitional shelters are not just a temporary post-disaster housing solution, but are a valuable asset which can be used by beneficiaries to further improve their lives. It is however important to note that the positive impact is only maintained when a household lives in the transitional shelter for a relatively short period, which means that transitional shelter should be a short-term housing solution. This is mainly because transitional shelters do not have enough facilities to allow a family to live in them for a long period of time. Follow-up research should tell us more about when this positive impact starts to decrease, and the maximum period that a family should live in a transitional shelter.

Living in a tent made life really hard for the entire family. There was no privacy and it caused a lot of stress. Receiving a transitional shelter has helped to get me and my family started again. It gave the family more comfort and relieved some pressure. My husband was more relaxed and more capable of taking care of the family again. – Ms. Isrijal, Banda Aceh.

Conclusion

The Vietnam case study clearly illustrates that basic quantitative research tools can be a valuable addition to personal stories and narrative reports. Quantitative analyses comparing beneficiaries and non-beneficiaries have served to substantiate and fine-tune assessments of the actual socio-economic impact of the housing intervention. The study on the Transitional Shelter programme in Aceh highlights the value of analysing and comparing different types of intervention. It has indicated some of the diverse and dynamic ways in which beneficiaries use the shelter they receive, and how the intervention impacted on their lives in the immediate and intermediate term. The study has generated valuable baseline data, which can be used for follow-up research. By applying a standardised methodology, comparison between programmes becomes a more realistic proposition. Standard long-term impact evaluations can also help organisations to develop a more sustainable shelter response strategy that corresponds to immediate and long-term needs.

Simone van Dijk and Alexander van Leersum are freelance consultants. Their email addresses are: vandijksg@gmail.com and alexander.vanleersum@gmail.com.

References and further reading

M. Bamberger, J. Rugh and L. Mabry, Real World Evaluations: Working Under Budget, Time, Data and Political Constraints, Sage Publications, 2006.

S. Van Dijk, A Socio-economic Impact Study on a Transitional Housing Program: A Case Study of a Red Cross and Red Crescent Society Housing Program in Indonesia, Eindhoven University of Technology, 2009.

A. Van Leersum, A Socio-economic Impact Study of a Post-disaster Housing Program. Case Study of a Red Cross Housing Program in Vietnam, Eindhoven University of Technology, 2008.

ODI, Improving Impact Evaluation Production and Use, ODI Working Paper 300, London, March 2009.