Evaluation of Immunisation Programme (EPI) in Lebanon (2013-2017)

"While vaccination rates across Lebanon continue to be relatively high, even among the various nationalities, data highlighted that there are still pockets of the population which hold considerable misconceptions about the safety and legitimacy of vaccines."

This evaluation report presents findings from a 6-month assessment of the Expanded Programme on Immunisation (EPI) component within the United Nations Children's Fund (UNICEF) Health and Nutrition Country Programme in Lebanon (2013-2017). The formative evaluation, which was commissioned by UNICEF Lebanon and conducted by Sayara Research, is intended to allow UNICEF to provide nuanced recommendations to strengthen the national immunisation system and future collaboration between the Ministry of Public Health (MoPH) and UNICEF.

The EPI in Lebanon works to ensure children from all socio-economic, gender, and ethnic backgrounds, especially the most marginalised and vulnerable, have access to routine immunisations. Given the significant increase in population in Lebanon in 2013-2015 as a result of the conflict in neighbouring Syria, UNICEF became a prime stakeholder ensuring the Lebanese government was capable of meeting vaccination needs of all children under the age of 5, regardless of nationality. Specifically, EPI in Lebanon aims to: elevate routine vaccination coverage per region of Lebanon to 95%, preserve Lebanon as a polio-free country (keeping in consideration the influx of Syrian refugees), and eradicate measles and rubella by the end of 2018.

The evaluation took a qualitative approach and aimed to assess UNICEF's support to EPI across Lebanon at national and localised levels through a desk review, 21 key informant interviews (KIIs), 14 focus group discussions (FGDs), 39 in-depth interviews (IDIs/exit interviews), and 32 structured observations of vaccination sites.

The evaluation team was focused on exploring 4 areas of interest, as prescribed by UNICEF Lebanon:

• Relevance - The evaluation team found that, while the support provided by UNICEF, as highlighted in the Annual Work Plans (AWPs), does address the immediate needs of the population, specific strategies to ensure that the most marginalised and remote had access to vaccines were limited. There was little mention of specific strategies to address families from lower socio-economic backgrounds or disabled families (with MoPH suggesting that disabled children were not within their immediate scope of work). Furthermore, reviews highlighted gaps in the overall incorporation of gender into EPI activities and strategies, with evidence that insufficient attention was paid to the nuances of gender relations and inter-household dynamics that can impact on the opportunity for female caregivers to ensure their children are vaccinated under EPI.
• Effectiveness - The team learned that "UNICEF provided invaluable support ensuring that not only the Lebanese populations had access to vaccines under the EPI, but also that migrants and refugees from neighbouring countries were not excluded....Nevertheless, a notable gap in efforts was the extent to which lessons learned across projects and monitoring activities were incorporated into UNICEF AWP and future interventions. Gaps were also noted in the extent to which UNICEF played a role in ensuring the effectiveness of EPI at local levels."
• Efficiency - "UNICEF teams were unable to provide considerable insight on costs associated with EPI activities in the AWP, apart from budgets noted within AWPs."
• Sustainability - "Data highlighted that MoPH currently have no alternative practices or strategies in place to cover the full cost of vaccines for the Lebanese, Syrian and Palestinian populations. Furthermore, the high turnover of staffing across the MoPH saw significant limitations in the extent to staff had necessary institutional knowledge."

One section of the report looks specifically at the relevance of strategic communications as a pillar of support provided by UNICEF. The AWPs from 2013-2017 highlighted that UNICEF intended to provide support to EPI through communication activities such as puppet theatre, children's games, FGDs, house-to-house mobilisation, and awareness sessions. While these activities were generally conducted through implementing partners, they attempted to address - at both national and local levels - the need to increase awareness and knowledge of the types of vaccinations required under the EPI vaccination schedule. Following the emergency response-based activities in 2013-2014 as a result of the conflict in Syria and influx of refugees and migrants, AWPs reflected the need for UNICEF to develop a more preventative focused and long-standing communication strategy to support MoPH in increasing the overall awareness of populations about EPI scheduling and the benefits of vaccinating children under the age of 5.

For example, at a national level, UNICEF's AWPs and interviews with the communications team highlighted that they occasionally supported the MoPH with national EPI-related campaigns. Secondary resources highlighted that in 2013, communication activities focused on 2 major disease outbreaks: measles in February and polio in October.

• After cases of measles was reported in Lebanon, at the request of the MoPH, UNICEF developed information, education, and communication (IEC) materials for a nationwide vaccination campaign, including 5,000 brochures for health workers, 10,000 posters for health centres, and TV spots and radio spots. Such materials were also provided to implementing partners to support their existing work in health-related activities. The names of implementing partners were not available to the vaccination team and not noted in secondary resources.
• A second national polio vaccination campaign was led in 2013 after polio cases were confirmed in Syria, with 1,150,000 brochures for parents, 6,000 brochures for health workers, 10,000 posters, and a TV spot and radio spot produced and broadcast nationally. UNICEF recognised the need to implement a community-based, tailored approach, so, in 2013, hakawete (traditional Syrian storytellers) were recruited to speak to Syrian refugee children and families in informal settlements about polio, promoting community knowledge and positive behaviour. Based on the needs identified by the MoPH regarding polio, efforts were intensified in 2014. One of the most notable media used was a 15-minute segment on LBCI's prime show 'Hki Jeles' to cover the polio campaign, with featured spokespeople from UNICEF and the MoPH. Twenty-five videos were produced and published, 35 photo missions were commissioned or conducted by communications staff, press events were held and supported, and 8 human interest stories were published. The success of the campaign was measured by an awareness rate of 86%, recorded in the most vulnerable areas, as well as by the 305 local, regional, and global media mentions of UNICEF Lebanon and the issues the most vulnerable children were facing.

Furthermore, at a more local level, field teams assessed the presence of communication material at public health centres (PHCs). All families that were present at the health facilities for the vaccination of their child reported they saw information about vaccines at the PHCs, and furthermore that such information was useful and informative. The presence of education material on EPI on the walls of the facilities was confirmed through site observations. Moreover, in Akkar, in the North and in the South, information specifically referring to UNICEF was reported. Among the types of information present in all locations were the names of vaccines provided and, in some facilities, the vaccine schedule. Yet the majority of health centres did not display any information on the benefits of vaccines or on how to administer vaccines.

Documentation and discussions on these areas, however, were not found during the evaluation period and therefore were not included as part of the performance of UNICEF. The evaluation team concludes that considerable effort has been made by various parties across Lebanon fill information gaps of the various populations and encourage families to vaccinate their children. While communication was regularly noted as a key area of intervention for EPI, apart from secondary material provided by UNICEF and highlighted in UNICEF reports, there was no primary data for the evaluation team to draw off that would have allowed them to directly credit the impact of communication activities to UNICEF.

Selected recommendations from the evaluation:

• The AWP should mention specific strategies to address families from lower-socio economic backgrounds. UNICEF could support the MoPH in the analysis of out-of-pocket expenditures related to immunisation and the drafting of a national policy on free access to vaccines and to vaccination till full immunisation of the child.
• In line with UNICEF global efforts, more should be done in Lebanon to work with the MoPH to ensure health professionals do not consider disability alone as a contraindication for immunisation.
• UNICEF can support PHCs through the introduction of practices and policies that promote equality across all nationalities.
• Capacity building activities should be more closely documented and should be focused on providing Training of Trainer (ToT) activities that support greater sustainability across MoPH, at the central, district, and local levels, so that UNICEF does not need to repeat training sessions.
• Building on UNICEF's global experience, UNICEF Lebanon could work more closely with MoPH to develop lessons learned approaches to their practices, with the introduction of workshops or seminars, encouraging shared dialogue.
• Given the role played by the private sector in achieving the objectives of the EPI, UNICEF could encourage the MoPH and support it in better understanding the provision of immunisation services by the private sector, including vaccine safety issues, whom the sector serves, how services are paid for, its contribution to coverage, and its integration with monitoring and surveillance.

The team concludes that "the findings of this evaluation and other lessons learned among UNICEF and MoPH need to be discussed and communicated to all relevant stakeholders. Interviews highlighted that many key stakeholders lack key information on the processes and practices of EPI. Efforts should be made to highlight and discuss all gaps, weaknesses and strengths of previous approaches and use these findings as lessons learned to create a more tailored and flexible approach to future AWPs."