Lessons from assessing the humanitarian situation in Syria and countries hosting refugees

In mid-2012, 18 months into the crisis in Syria, most actors agreed that the picture of the humanitarian situation was incoherent and fragmented: displacement flows, the scope and depth of humanitarian needs and the longer-term impact on infrastructure and livelihoods were largely unknown. Much of the problem revolved around the sensitivities associated with gathering and sharing information on the affected population and restricted access to the field. In neighbouring countries hosting hundreds of thousands of refugees, responses diverged and were not based on a coordinated and harmonised needs analysis.

The Syria Needs Analysis Project (SNAP) was established in December 2012 to strengthen shared situation awareness among humanitarian actors responding to the Syrian crisis. SNAP conducts independent analysis of the impact of the crisis in consultation with stakeholders in the region, providing regular briefings that focus on prioritising needs, lessons learnt and assessment and information gap analysis. Available to all, SNAP’s reports promote inter-agency information exchange and joint needs analysis.

In addition, SNAP provides technical services to humanitarian actors. In early 2013, SNAP supported three inter-agency assessments in northern Syria by providing technical input on methodology design, training, analysis and reporting. In August 2013, SNAP consulted with various humanitarian actors to capture some of the lessons identified to date in undertaking assessments in the Syrian context, within Syria and in Lebanon and Jordan. This paper summarises the key findings and recommendations.

Key findings

• There is a lack of coordination of assessment activities amongst humanitarian actors.
• Assessment fatigue is an issue, especially where delivery of assistance is limited or non-existent.
• Political and personal sensitivities hamper data collection.
• The translation of questionnaires and methodology is challenging and time-consuming.

Recommendations

While the multiplicity of actors, security constraints and a continuously evolving humanitarian situation present many challenges, three overarching recommendations can be drawn from the experiences to date to improve assessment practice:

• Assessment Working Groups in each country should actively encourage and foster a culture of coordination by:
  • Agreeing on key information needs to be included in every assessment.
  • Encouraging the sharing of assessment plans, data and findings, at least amongst participating organisations.
  • Promoting and facilitating joint or harmonised assessments, wherever possible.
• The establishment of countrywide monitoring systems would also contribute to a shared understanding of trends and patterns and reduce assessment fatigue among the population.
• Assessment methodologies, international standards and questionnaires should be adapted to the local context, and should have regard to the respondent and the enumerator’s safety. When formulating questionnaires, training enumerators and analysing results, careful attention should be paid to the nuances of Syrian Arabic compared to that spoken in neighbouring countries.

Findings

Coordination

A multitude of actors are operational in the region and, whilst the quantity and scope of assessments continue to increase, coordination of these assessments is too often inconsistent. Furthermore, not all assessments follow international standards; there is little joint analysis of results at a sector-working group level; and information is often not shared in a timely manner. Consequently, the information available is often patchy, it is difficult to make comparisons between different sets of information and information does not contribute to a country or region-wide picture of needs. Potential explanations for this lack of coordination include concerns about sharing data for security reasons; the challenging environment, which results in resources being focused on response rather than coordinated assessments; and a lack of dedicated experts in the region.

This lack of coordination contributes to assessment fatigue – an issue both within Syria and in host countries, especially where the organisation undertaking the assessment provides no visible assistance. In Jordan, assessment fatigue has led to a significant number of refugees refusing to participate in assessments. However the highly dynamic situation makes frequent assessments necessary. Increased sharing of information, joint and inter-sectoral assessments and combining assessments with aid interventions were mentioned as possible ways to relieve the burden of assessments on the population.

Assessing the needs of the Syrian population

Although the humanitarian situations in Syria and refugee hosting countries vary significantly, some issues are relevant region-wide. A major task is ensuring that enumerators are able to communicate clearly with respondents and record the data so as to accurately convey the information that the assessment is designed to collect. The communication challenges identified in the Syrian context fall into two categories: respondent sensitivity and language use.

Syrian communities are often reluctant to share information due to the highly political or personal nature of certain topics. Minimising the risk (perceived or real) to respondents when taking part in an assessment is essential to encourage participation. Any methodology must therefore clearly frame the assessment as humanitarian, and avoid, as far as possible, any political connotations. Designing the questionnaire so that it does not elicit politically sensitive answers; training enumerators in humanitarian codes of conduct; and clearly introducing the assessment to the respondents will help minimise the perceived threat to them. For instance, organisations assessing protection concerns have avoided questions related to the perpetrator of a violation, so as not to jeopardise the security of the enumerator or the respondent.

Discussing sexual and gender-based violence in one-on-one surveys with Syrians is extremely challenging, particularly if the agency concerned does not provide related services. One organisation recommended that, when working on protection issues, agencies should train service delivery staff and those with existing relationships with interviewees to conduct surveys, rather than training enumerators on protection issues, to maximise the quantity and quality of information shared. Hygiene-related questions, including how many times hands are washed or people shower per week, can also be sensitive as such questioning might infer poor hygiene practice in a culture that values high standards of hygiene. The lack of legal status in a host country, registration with the UN High Commissioner for Refugees (UNHCR), relationships with host communities, areas of origin for IDPs and the power structures within IDP camps are all highly sensitive topics.

The translation of questionnaires and methodologies from English into Arabic was mentioned several times as an issue requiring careful attention. As the Arabic spoken in Jordan and Lebanon differs from Syrian Arabic, a translator familiar with Syrian Arabic and the humanitarian vocabulary is required. Not only can the nuance of a question get lost in translation but definitions of key terms used within Syrian communities sometimes differ from definitions used by international organisations. Examples of terms requiring special attention include:

• ‘Household’: Syrians often live together with not only their nuclear family but also part of their extended family (uncles, brothers, the elderly). Hence, the specific ‘humanitarian’ meaning of the word ‘household’ or ‘family’ should be clearly explained to the respondent at the start.
• ‘Orphan’: in the region, an orphan is culturally recognised as a child who has lost his father, not necessarily a child who has lost one or both parents.
• Child labour and child marriage are common in Syria and families may not recognise these as issues. Furthermore, some families may deny their occurrence for fear of losing assistance or out of embarrassment over not sending their children to school.
• Estimation of averages: questions related to the average time or money spent on activities or goods are not always understood. Instead of working with averages, enumerators should specify a timeframe such as ‘this week’.
• Protection terminology is difficult to translate. Some standard operating procedures have been established in Jordan, yet some misinterpretation is to be expected as there is debate among Arabic speakers on the meaning of terms involved.
• Within Syria, the concept of rights, including human rights and child rights, is very sensitive because of the Syrian government’s restrictive approach towards human rights. The word ‘rights’ should therefore be avoided during any assessment in Syria to avoid endangering enumerators or respondents.

Assessment methodology – in neighbouring countries

A range of issues should be considered when designing any assessment in the countries neighbouring Syria. Humanitarian actors should reflect on the type, scope and timing of coordinated assessments on the situation in each country. Designing a situation monitoring system should be explored at district or sub-district level in affected areas to follow up on humanitarian conditions, trends and patterns over time, provide strategic information and trigger more in-depth or targeted assessments as necessary. Distinction should be made between the various groups making up the humanitarian profile to allow for comparison of needs between those groups (i.e. refugees in host communities versus refugees in camps). All assessments should be cleared by the host government and enumerators provided with proof of government approval. Visiting local governance structures before assessments are undertaken is also important – in Lebanon, local municipalities are not always notified of nationally approved assessments.

Identifying the vulnerable amongst refugees and host communities in non-camp settings is very difficult and there is no accepted methodology to identify and sample these families. In Lebanon, identifying Syrian key informants able to represent or speak directly on behalf of the refugee community has been found to be a challenge, partly because of the diffused displacement of refugees. In Jordan organisations suggest working with key informants and local organisations as vulnerable Jordanians among host communities may refuse to participate, being too proud to identify themselves as vulnerable.

Enabling the participation of all vulnerable groups is also challenging. Although Syrians do not seem to ‘hide’ disabled family members, they will speak on their behalf. Similarly, male household members often talk on behalf of female members. Hence, mixed-gender survey teams should be used to speak with minority groups directly, wherever possible. Syrian minorities, such as Christians, Assyrians and Kurds, are also difficult to identify and might not be willing to identify themselves as part of a minority group.

Security arrangements should be in place prior to any data collection. Some organisations in Jordan only undertake assessments in guarded public facilities, such as distribution sites, schools or health facilities, to reduce the risk of security incidents. This, however, may make interviewees more reluctant to speak openly: one organisation in Lebanon reported that Syrians speak more freely when assessed within their homes rather than in public. Conversely, while people are hesitant to share personal information, the deteriorating humanitarian situation increasingly forces them to seek assistance. Thus, some respondents may exaggerate their vulnerability.

A number of tools and assessment methodologies have been successfully adapted in the region. However, several issues remain: there is no accepted methodology to identify and sample (often invisible) vulnerable groups, including unregistered refugees; in Lebanon, the government and UNHCR are still working to harmonise unique identifiers for administrative areas; and there is no consensus concerning the types of employment categorised as child labour as it depends on different variables and varies by country. This needs to be clearly indicated in any assessment.

An educated workforce is available in the region and it is relatively easy to find and train enumerators. However, most need training in humanitarian principles, jargon and assessment ‘etiquette’– explaining to people the purpose of the assessment, being respectful and not entering someone’s home unannounced. Moreover, there are local institutions that could support assessments, including the Central Bureau of Statistics of Lebanon and the American University of Beirut. Such bodies can not only provide enumerators, but also help with assessment contextualisation and results analysis.

Assessment methodology – within Syria

In Syria, the complexity of the situation, the dynamism of population movements, the limited time validity of the information collected and difficult access to the field (government restrictions and security constraints severely hamper humanitarian assessments, especially at the household level) significantly inhibit assessments. This calls for a differentiated approach in those areas where assessments are possible and those areas where access is more restricted, but where there is still a need to capture main trends and patterns. Assessments undertaken to date have made use of key informant interviews, focus group discussions and remote assessment practice (questioning new arrivals in host countries on the situation in the area they recently fled).

Civil society actors in Syria have the capacity to undertake assessments and should be involved because of their contextual knowledge and ability to access different areas. The importance of involving Syrian organisations in assessments was highlighted by multiple actors. Facebook, e-mail and Skype are widely used in Syria with limited awareness of the related security risks: appropriate security measures should be in place when communicating with individuals inside Syria. As for assessments in host countries, enumerator teams should combine residents and non-residents of the area assessed. Locals can better identify endemic socio-economic problems, political dynamics that may not be immediately evident and access areas; non-locals help ensure objectivity.

Nic Parham is the Assessment Expert and Project Lead for the Syria Needs Analysis Project (SNAP). Leonie Tax, Lynn Yoshikawa and Kevin Lim are SNAP Information Analysts.

Key resources for undertaking an assessment

SNAP, Needs Assessment Lessons Learned: Lessons Identified from Assessing the Humanitarian Situation in Syria and Countries Hosting Refugees, September 2013, www.acaps.org/en/pages/syria-snap-project.

ACAPS, Coordinated Assessments in Emergencies. What We Know – Key Lessons from Field Experience, November 2012, www.acaps.org/en/resources.

CARE, CARE Emergency Toolkit: Assessment, CARE International, Geneva, 2009, http://careemergencytoolkit.org.

IASC, Operational Guidance Note for Coordinated Assessments in Humanitarian Crises, 2012, https://docs.unocha.org/sites/dms/CAP/ops_guidance_finalversion2012.pdf.

ICRC, Professional Standards for Protection Work, 2013, www.icrc.org/eng/assets/files/other/icrc-002-0999.pdf.

ECB/ACAPS, The Good Enough Guide to Assessments (Draft), January 2013, http://www.acaps.org/resourcescats/downloader/gega_draft_1_0/148).

IFRC, 2008, Guidelines for Emergency Assessment, http://www.ifrc.org/Global/Publications/disasters/guidelines/guidelines-emergency.pdf.

The Sphere Handbook, 2011 edition, www.sphereproject.org/handbook.

World Food Programme (WFP), Emergency Food Security Assessment Handbook (Second edition), http://home.wfp.org/stellent/groups/public/documents/manual_guide_proced/wfp203246.pdf.

World Vision International, Emergency Capacity Building Project, Impact Measurement and Accountability in Emergencies: The Good Enough Guide, 2007, http://www.globalpolicy.org/images/pdfs/0209goodenough.pdf.