Conducting simulated field visits for insecure locations in Somalia

With two decades since the central government in Somalia collapsed and despite efforts to support stabilisation, several parts of the country are consistently viewed as among the most dangerous environments for aid workers. Against the backdrop of a volatile security situation is a chronic humanitarian crisis where food prices, livestock survival and water and food availability are constantly under stress from drought and armed conflict. Between October 2010 and April 2012, Somalia was at the heart of a drought crisis in the Horn of Africa, affecting 13 million people and causing an estimated 258,000 excess deaths. F. Checchi and C. Robinson, ‘Mortality among Populations of Southern and Central Somalia Affected by Severe Food Insecurity and Famine during 2010-2012’, FSNAU, 2013.

While the humanitarian response to this crisis ended in 2012, people continue to face drought, violence and food insecurity. According to the Food Security and Nutrition Analysis Unit (FSNAU) for Somalia, FSNAU, ‘Somalia Post-Deyr 2015/16 Food Security and Nutrition Outlook (February to June 2016), 17 February 2-16, http://fsnau.org. 38% of the population of Somalia are acutely food insecure and 304,700 (12%) of children 6–59 months old are acutely malnourished. Biannual surveys by FSNAU show that, in many parts of South and Central Somalia, the situation remains in the ‘critical’ category for malnutrition in the FSNAU’s ranking system, just below the ‘crisis’ classification used during the famine. In much of this part of the country government presence is limited and basic services are delivered by NGOs.

In response to the humanitarian situation in Somalia, Save the Children has long-standing programmes to prevent and treat acute malnutrition, provide health care services, improve water and sanitation and enhance household food security and livelihood options for communities. Due to the very high rates of acute malnutrition, Save the Children’s programming includes Community-based Management of Acute Malnutrition (CMAM) programmes in Awdal region, Somaliland, Karkar and Nugaar regions in Puntland and Hiran, Baidoa and Banadir regions in Central and South Somalia, supporting 96,415 children in 2015. In accordance with the national protocols for Somalia, all children 6–59 months old are screened by community volunteers who take their mid upper arm circumference (MUAC) to determine if they require treatment. Severely acutely malnourished children are referred to the nearest health centre, where they receive a medical check, and unless they have serious medical complications they are treated as outpatients with follow-up once a week, where they receive routine medical treatment and Ready-to-Use Therapeutic Food (RUTF). The Outpatient Treatment Programmes (OTPs) are run by Somali national staff at the health centres, with routine supervision from Somali national programme managers. International nutrition technical advisors conduct regular support visits where security allows.

While all Save staff have direct access to programme areas in Puntland and Somaliland, access to sites in Central and South Somalia is limited, especially for non-Somali staff, and in some cases staff who are not from that specific implementation area and ethnic group. This can result in a lack of understanding by non-local staff of the situation on the ground and the quality of the programme. At the same time, lack of contact with technical staff can affect motivation and opportunities for learning from field support visits. Save the Children has explored different methods for remotely monitoring and supporting field teams: Simulated Field Visits (SFV) are one of these methods. The SFV has been developed to enable monitoring of nutrition programmes, including assessing programme performance against established standards, identifying bottlenecks and challenges and providing a connection with field teams to foster motivation and capacity development.

The first SFV was conducted in 2013 as part of a review of a nutrition programme in Puntland and Hiran by headquarters and regional staff ahead of a donor audit. The review of the Puntland programme was conducted without much access difficulty, but insecurity prevented direct access to Hiran. In place of a face-to-face review as in Puntland, the team in Hiran was asked to take photos of key treatment points in the OTP sites, such as screening and measurement-taking, appetite testing and the dispensing of RUTF and medicines, and of children having their MUAC taken; scan or take photographs of a sample of patient cards; and share supply monitoring figures/sheets and reports and supervision checklists. Skype calls were also conducted with the programme manager and OTP staff to get feedback on challenges and progress.

The review of patient cards and stock reports provided significant insight into the level of understanding of the treatment protocol by implementing staff and resulted in a number of recommendations for improvements. Reviewing photographs helped to visualise the programme, but the quality of the pictures was often poor. However, even with low-quality photographs it was possible to identify possible areas for improvement in site organisation and measurement-taking. The review team felt that the process had significantly increased understanding of how well the Hiran programme was operating, where there were difficulties and what to prioritise in terms of programme improvement. A comparison with the Puntland programme, which had received a lot of on-site technical support, showed large differences in programme quality, indicating a need for much more regular support of the Hiran programme. The method was then standardised to include documentation, a technical and joint review of documents, feedback and action planning.

Documentation

The same set of documents and photos as for the 2013 review have to be requested for all SFVs. However, to ensure that the correct photos are provided specific guidance in the form of a checklist has to be followed by field teams, detailing which parts of the site to photograph and the distance from which photos are to be taken. Photographs should include:

• Outside of the OTP from ten metres away.
• Latrines (if available).
• Seating/waiting area for caregivers.
• Photo showing roof/shading of the waiting area (if there is any).
• Stock/storage room (if this exists).
• Photos of how the RUTF and medicines are stored in the room.
• Photo of drinking water storage.
• Children and pregnant and lactating women having their MUAC taken.
• Nutrition/IYCF education sessions.

Technical review of documents

For the purpose of SFV, minimum standards were set in terms of site set up, equipment, waiting space, times and supplies and storage of therapeutic foods and medicines as well as water sanitation facilities. Submitted documents have to be reviewed against these standards. To ensure that standardised checks are made on patient treatment cards an audit tool in Excel was developed, where metrics from the review of the cards are entered. These look at whether correct processes were followed at admission and discharge, the amounts of therapeutic food and medicines given and data management. Photographs and checklists are reviewed against the minimum standards set for the programme.

Joint review of documents: discussion with the field team over Skype

During the joint review of documents, calls to the field team are conducted to go through the documents and photos and discuss findings. Where screen sharing is possible it is used.

Feedback and action planning

After the SVF, the review team has to write a report including findings and actions agreed upon. The programme manager then leads the field team in implementation of the agreed actions. The next SVF then compares its findings with the previous visit to track improvements and address recurring issues.

Implementation

The SFV process was refined over the course of implementation between February 2014 and October 2015. The October review was the first to test the final process. During this review, it was also possible for the first time, due to a stable internet connection, to jointly review photos and documents with the team in Hiran via Skype’s video and screen sharing function. Following each review a report is produced and shared with the field team, and follow-up actions are drawn up jointly by the review team and field staff who participated in the SVF.

Initial findings: what improved?

Early indications are that this process has helped improve programme quality. The second SFV in February 2014 revealed improved adherence to admission protocols and documentation of beneficiary information, with more than 90% of children admitted correctly, given medical checks and provided with appropriate medication. However, in some cases, certain dosages of RUTF were still incorrect. No information on actions taken for children with static and/or faltering weight were provided for a third of beneficiaries (31%), and information on whether the child was vaccinated against measles was missing for all of the patients. Nearly half of the cards (49%) were found to have no information on discharge outcomes. Some photographs indicated that MUAC was not taken correctly, and the waiting space and facilities were inadequate.

Follow-up SFVs showed improvements, with 61%, 74% and 96% of the eligible children provided with measles vaccination and/or vaccination status noted in the second, third and fourth reviews respectively, compared to none in the first review. Children who received the correct amount of RUTF as per their weight improved from about 70% in the initial review to 100% in the fourth. Correctly recording discharge had also improved from 51% in the initial review to 80% in the last review. Ensuring that poor weight gain is identified and investigated showed less progress, so this will be an area of focus for training and future SFVs.

The process has received positive feedback from both technical staff based in Nairobi and the team in Hiran, and it will be continued and expanded for insecure locations elsewhere in Somalia. It is time-consuming, with a lot of preparation needed by field staff, but as it is intended to replace time spent on actual field visits this is felt to be acceptable. Where internet connections are strong this can be a participatory approach, which aids acceptance of the final recommendations. Where there isn’t a strong internet connection the process is still possible, but documents and photos need to be physically transferred to the review team, so additional time must be factored into the process.

Much of the analysis depends on information provided by field teams, so it would be possible to ‘stage manage’ visits, provide falsified beneficiary cards and create a false impression of the programme. However, the effort involved in doing so would be similar to that required to actually improve the programme, and it is felt that, if this takes place, even ‘faking it’ may lead to an improved understanding of how the programme should be managed.

Next steps

Save the Children’s Somalia programme will continue to use this method and conduct quarterly SFVs for inaccessible locations where there are nutrition programmes. Save the Children is also using mobile phones to support early warning and supervision in Somalia, and will look at incorporating the platform into the SFV. Remote monitoring systems elsewhere J. Bauer, A. Mouillez and A. Husain, ‘Not a Rolls-Royce but It Gets You There: Remote Mobile Food Security Monitoring during the Ebola Crisis’, Humanitarian Exchange, no.64, June 2015, https://odihpn.org  have demonstrated the potential of text messages (SMS) and interactive voice response (IVR – pre-recorded audio messages) for collecting information from difficult-to-reach populations. Save the Children is piloting the use of SMS to collect feedback from CMAM beneficiaries on the quality of services. Data collected from this system will be included in SFV to supplement information gathered and presented by programme staff.

While SFV appears to have had a positive effect on programme quality, it is difficult to determine from the current data exactly how effective this method has been. Save the Children will further investigate the effectiveness of SFV for improving OTP quality and staff motivation in Somalia. Subsequently, the possibility of using this for other sectors or in other locations where access can be difficult, such as Syria and South Sudan, will be explored.

Monica Zikusooka is the Head of Monitoring, Evaluation, Accountability and Learning for Save the Children Somalia. Zinet Nezir Hassan is an independent consultant. Alison Donnelly is the Humanitarian Nutrition Advisor for the East Africa region for Save the Children. Rachel Mose is a Save the Children Nutrition Specialist for Somalia.