In-Depth Evaluation of the Reaching Every District Approach in the African Region, 2007

Since 2002, the World Health Organization (WHO) and partners have supported the Reaching Every District (RED) approach to improving immunisation coverage. Developed in response to the decline in immunisation systems performance in the African Region in the 1990s, RED focuses on building the capacity of districts and health-facility-level health workers to initiate and strengthen each of the following five components: planning and resource management, supportive supervision, outreach, linking with communities, and monitoring for action. This evaluation report covers the socioeconomic and health systems contexts within which immunisation programmes operate, the degree of implementation of each of the five components of RED, and the major lessons learned while scaling up the approach.

The RED approach is designed to contribute to the goals of the Global Immunisation Vision and Strategy (GIVS) and Millennium Development Goal (MDG) #4, which calls for the reduction of child mortality by two-thirds by 2015. In 2005, the WHO and partners conducted a rapid assessment of RED implementation in five countries. This more comprehensive evaluation, conducted in 2007, reviews the status of RED implementation to determine progress toward improving immunisation services and coverage. The evaluation's results will be used to further strengthen routine immunisation (RI) and sustain past coverage gains. The evaluation was conducted over a one-year period and carried out in three phases: a desk review, nine country visits, and regional analysis and report writing. Twenty-seven countries provided district coverage data for the desk review; nine of them were selected for in-depth evaluation: Benin, Cameroon, Democratic Republic of the Congo (DRC), Ethiopia, Ghana, Madagascar, Sierra Leone, Togo, and Uganda. Standardised questionnaires were used at national, district, and health facility levels. A total of 68 districts and 133 heath facilities (HFs) were visited.

In short, the 2007 evaluation found that districts implementing the approach were raising immunisation coverage, and that many health facilities were including community mobilisation, meetings with community leaders, and frequent interactions with community volunteers in their work plans and regular activities. The evaluation also confirmed that community volunteers were commonly involved in providing health education, assisting during outreach sessions, and tracking immunisation defaulters. Fewer than half of all community volunteers, however, were registering pregnant mothers and births, and volunteers were not sufficiently involved in planning immunisation sessions and/or reporting diseases. The authors conclude that "active involvement of community volunteers [is] key in increasing immunization demand, particularly for outreach in rural areas."

A more detailed summary of key findings follows:

• Introduction and scaling up of RED: Countries differed in how quickly they expanded RED, with a few launching nationally and others prioritising and phasing-in new districts more gradually. By 2006, 90% of districts in the nine countries had introduced RED. In 80% of districts, outreach was the component mentioned most often by staff in describing what RED meant in their districts. The most notable changes since the introduction of the RED approach, as noted by district staff, were additional outreach sites and community meetings.
• Planning and resource management: Only half of district microplans indicated hard-to-reach populations and plans for reaching them. Catchment area maps were found in more than two-thirds of HFs, but they commonly lacked details. Stockouts of measles or Diphtheria-Pertussis-Tetanus (DPT) vaccine occurred within the last year at the district and HF levels in all but one of the nine countries. Training and managerial capacity gaps exist in many districts.
• Supportive supervision: The majority (65%) of districts included supervision in their annual work plans. However, only about a third of districts reported receiving supervision visits from the national level in the preceding three months. Visits from districts to HFs occurred more frequently (70% of HFs reported visits in the preceding three months). Supervisory checklists were commonly used (82% of HFs) and supervision of the Expanded Programme on Immunisation (EPI) was integrated with that of other health services in six of the nine countries. While the majority of HFs (78%) reported receiving immediate verbal feedback, written supervisory feedback was documented in only about a third of facilities. District review meetings occurred in almost all districts, providing opportunities for training which may be under-utilised for this purpose.
• Outreach: Most countries reported an increase in the number of outreach sessions planned and held since the introduction of RED. Although a majority (62%) of HFs reported conducting >80% of planned outreach sessions, less than a third of districts were able to show catchment area maps indicating outreach sites. HF staff in approximately one-third of facilities reported that communities were involved in outreach session scheduling. However, staff in approximately one in four districts and facilities reported that they had not yet reached all communities with fixed, outreach, or mobile sessions, mostly due to limited transportation and/or inadequate numbers of health personnel.
• Community: Community volunteers were active in all countries, typically assisting with defaulter tracking, outreach, and community education. Two-thirds of HFs reported holding regular meetings with the community, and 74% of facilities reported that community volunteers had been trained to assist with immunisation activities. Approximately half of HFs with community volunteers reported providing some form of incentive (such as training or transportation reimbursement). Increases in the number of community meetings were also reported.
• Monitoring for action: Immunisation monitoring charts were displayed in the majority of districts and HFs, and most were correct and up-to-date. Health workers in 70% of those facilities with monitoring charts were able to explain their facility's performance. Ninety-five percent of districts reported conducting review meetings with their HFs during the past year, at which time immunisation data were discussed. In half of the HFs, staff perceived that denominator estimates were inaccurate. Fewer than half of the HFs had methods other than immunisation registers for tracking defaulters.
• Integration: Provision of immunisation with other maternal and child health services (Vitamin A supplementation, family planning, antenatal care, bednet distribution, deworming, growth monitoring, curative care) was common in HFs and outreach sessions.
• Financing RED and RI: Countries have used national and district health budgets, WHO and United Nations Children’s Fund (UNICEF) grants, in-kind support from non-governmental organisation (NGO) partners, and GAVI Immunisation Services Support (ISS) funding.
  
• Access and utilisation: Although not necessarily attributable to RED, DPT1 coverage increased from an estimated 69% of children under one year of age in 2002, to 87% in 2006. Utilisation, as measured by DPT3 coverage, also increased from 57% to 79% over the same period.
  

Recommendations:

The evaluation team recommends that the global and regional partners of national immunisation programmes:

1. Revise the RED approach guidelines to include specific standards and indicators for all five RED components.
2. Support a regional or multi-country forum on RED and RI strengthening to share lessons learned, best practices, and challenges.
   
3. Support countries to document the costs and to advocate for new funding for RI.
4. Support WHO/Africa Regional Office (AFRO)'s ongoing efforts to define and determine the impact of integrating immunisation and other priority health services.
5. Continue to work across countries to improve the availability and use of reliable district coverage data.

The evaluation team recommends that countries and their national immunisation programmes:

1. Continue reinvigorating outreach, improving community linkages, and increasing immunisation coverage.
   
2. Ensure that all districts implementing RED are provided sufficient funding, training, and technical support.
3. Systematically assess and address reasons for stockouts.
4. Ensure adequate planning for supervision, improved feedback mechanisms, and sufficient resources for the transportation and allowances of supervisors.
5. Work with districts and facilities to analyse and address the reasons why up to one-quarter of HFs are failing to reach all their communities.
6. Evaluate and continue to strengthen linkages between HFs and communities. The opinions of communities and families should guide future changes in immunisation services. Document lessons learned working with community volunteers and develop standard tools for defaulter tracking and community education.
7. Improve the quality of immunisation data and encourage its regular analysis and use. Continue to hold regular review meetings at district level. Report separately on vaccinations given at fixed and outreach sessions and track the integrated delivery of immunisation and other priority services.

"Although regional immunisation coverage has increased steadily since 2002, intensified efforts are needed to achieve the ambitious GIVS immunisation coverage goals and the MDG mortality reduction goal. The RED approach can be an important tool for addressing immunisation performance and strengthening the district management of immunisation and other health services."