Collaborative Community Checklists for Immunisation: A Feasibility and Acceptability Study in Rural Myanmar

"Before they thought that immunization was totally the responsibility of the sayarma (Midwife) and now they regard it as a whole of community responsibility. All all...parents...community...now they understand that to achieve success in immunization, they all need to cooperate." – Midwife

International research suggests that improved community engagement is key to addressing vaccine hesitancy and increasing coverage. Operational research suggests that one such tool for engagement, community scorecards, can increase health service transparency and accountability and can help build collaborative relationships between health providers and communities. In 2016-17, the Burnet Institute collaborated with the Myanmar Expanded Programme on Immunization (EPI) to explore the feasibility and acceptability of a new approach to improve community engagement with and uptake of immunisation in hard-to-reach communities as a complement to continuing work to improve immunisation service quality and access. This new approach is called the Collaborative Community Checklists for Immunization (CCCI) project and utilises both community- and provider-oriented immunisation quality checklists.

Funded by 3ie, the International Initiative for Impact Evaluation, with support from the Bill and Melinda Gates Foundation, this formative evaluation was conducted in 3 rural villages of Ngaphe Township, Magway Region, Myanmar. The primary aim of the study was to test the feasibility and acceptability of the CCCI intervention. In addition, the researchers sought to assess community and immunisation providers' perceptions of the effect of the intervention on: community immunisation knowledge; immunisation service uptake; and whether the intervention enables communities and providers to work together to identify and address local barriers to immunisation.

The World Health Organization (WHO) has developed service quality checklists for use by health providers in surgery, safe motherhood, and, most recently, immunisation. The team modified the WHO tool and added a new community-based version, aiming to synchronise quality improvement in clinics with community education to engage families through a better understanding of good-quality service provision. The CCCI modification comprised the following components:

• Through a participatory process, the study team worked with immunisation providers (midwives) and health officials to adapt the wHO immunisation session checklist to the local context. Providers were trained and supported to use this provider checklist during immunisation sessions.
• The study team held design workshops to support 164 adult community members, with an emphasis on pregnant women, parents of young children, and Village Health Committee (VHC) members, in 3 villages to identify barriers to immunisation uptake and design a checklist to assess quality of immunisation service. They drew in part on new communication materials from the national immunisation programme to provide a reminder of the immunisation schedule. Caregivers were encouraged and supported to use this community checklist during immunisation sessions.
• Community members were assigned to be volunteer checklist assistants and were trained to provide support to caregivers in using the community checklist at immunisation sessions. These assistants also collated feedback from completed checklists and collaborated with community members and midwives to discuss any issues raised through the checklist.
• The study team conducted monthly health education sessions in each study village to build caregiver understanding of quality immunisation services and educate them on use of the community checklist.

The underlying theory of change for this intervention is that by using locally adapted immunisation session checklists, caregivers and midwives would work holistically towards improving overall immunisation outcomes for children. In particular, the intervention would lead to an increase in caregivers' knowledge on immunisation, increase the demand for quality services, and improve communication between caregivers and midwives. In addition, the use of provider checklists combined with feedback from the caregiver checklist would help midwives improve their service delivery. The improved quality of services and increased demand would lead to more children getting fully immunised, which in turn would reduce the prevalence of vaccine-preventable diseases (VPDs).

Implemented in 2016 and 2017, the mixed-methods study in Ngaphe Township entailed: (i) qualitative methods (e.g., focus group discussions (FGDs), key informant interviews (KIIs), and in-depth interviews, or IDIs) to explore knowledge of and attitudes towards immunisation, including the elements of good-quality services, barriers to and enablers of immunisation, and acceptability of checklist use; and (ii) quantitative data collection methods to assess supply-side determinants of quality and access to services, immunisation rates, usage of community checklists by caregivers, and immunisation provider experiences of checklist use.

A total of 52 and 107 community members and immunisation providers participated in qualitative data collection at baseline and endline, respectively. At endline, a total of 133 parents of children aged 2 years or less and pregnant women participated in a community survey. Following preliminary analysis of qualitative and quantitative data analysis, field staff facilitated community and individual meetings in each project village to discuss preliminary findings with study participants.

The overall findings indicate that the intervention is feasible in rural and low-resource settings. Eighty-four percent of surveyed caregivers used the checklist to assess immunisation services at least once. Caregivers reported that the community checklist was quick and easy to use and was not overly burdensome.

Furthermore, all midwives in the study area had used the provider checklist in planning and delivering immunisation services. They reported that using the checklist was relatively easy, and that there was increased recall regarding items to be prepared and actions to take before, during, and after an immunisation session.

Caregivers, midwives, and community members reported a number of benefits associated with the intervention, including perceived improvement in: the community's knowledge regarding immunisation, communication between providers and the community, attention to elements of immunisation service quality, and care-seeking for childhood vaccination. Overall, mothers and fathers participating in endline FGDs identified increased caregiver commitment to vaccination as the most significant change resulting from the intervention.

Among the challenges that were encountered during this formative work: High levels of caregiver illiteracy made it difficult for some caregivers to complete the community checklist, and they needed help from checklist assistants. (While the original plan was to have a fully pictorial community checklist, time and resources made this impossible to achieve.) Further development of the CCCI intervention, including changes to the format of community workshops and modification of the community checklist for low-literacy caregivers, could boost timeliness and other outcomes.

A large dissemination workshop was held in the national capital on May 24 2017, at which preliminary results were presented and their implications discussed. Participant and stakeholder suggestions for intervention improvement were generated and taken into account by the study team, who suggests that the following modifications to the CCCI be made:

• Harmonisation of any provider checklist with other supervision approaches in use in Myanmar;
• Adaptation of community checklist processes and tools so they focus on improved understanding of the immunisation service and the family's role in actively supporting vaccination;
• A continued commitment to regular, participatory community health education to build basic immunisation knowledge, increase understanding of the community's role in supporting routine services, and increase awareness of their right to a quality vaccination service; and
• Greater emphasis on regular, participatory community discussion of community checklist data in order to identify ways that community members can further support immunisation services and to foster collaborative action by community members and providers in addressing local immunisation barriers.

The study team concludes that a full impact evaluation of a modified CCCI approach to support immunisation strengthening is warranted, given this demonstration of feasibility, acceptability, and potential benefits in improving both community engagement and quality of care.