Impacts of Community-Led Video Education to Increase Vaccination Coverage in Uttar Pradesh, India: A Mixed Methods Randomised Controlled Trial

"...findings highlight the need for both supply- and demand-side interventions: the benefit of strengthening community health worker engagement and outreach, and the importance of localized knowledge of the social context in SBCC [social and behaviour change communication]."

In rural Uttar Pradesh, India, children and their caregivers face multiple barriers to vaccination, one of which is low awareness of and demand for vaccination. This study, carried out by PATH with funding from the International Initiative For Impact Evaluation (3ie) and the Bill & Melinda Gates Foundation, evaluated whether and how a community-led video education intervention could increase demand for vaccination, increase vaccination coverage among children 6 to 17 months, and effect change in the knowledge, attitudes, and behaviours of their caregivers. The entire project, including intervention and evaluation, was implemented from July 2016 to January 2018 in Airayan and Hathgaon blocks of Fatehpur District in Uttar Pradesh.

PATH's Projecting Health (see also Related Summaries, below) is a multi-component intervention package, implemented since 2012, that consists of:

• Design and production of culturally appropriate, "hyper-local" videos with tailored behaviour change communication (BCC) messaging to target root causes of suboptimal health care–seeking behaviours. These videos are co-created by a local community-based organisation, Nehru Yuwa Sangthan-Tisi (NYST), village women, and government-supported frontline health workers (accredited social health activists, or ASHAs). The latter are female community members who work in the villages where they live on health promotion and mobilisation activities. For their participation in intervention activities, ASHAs were provided with training, a stipend, and recognition.
• Video screening by ASHAs at: mothers' group meetings; Village Health and Nutrition Days (VHNDs), which are often attended by expecting and lactating mothers, their mothers in-law, and sisters in-law; evening screenings for men; and in-home screenings for mothers from hard-to-reach (HTR) or other vulnerable communities. (The project originally intended for ASHAs to load the videos on their mobile phones to share with community members, which would have allowed for greater informal dissemination and repetition of key messages when videos were re-watched at home. This did not occur due to the lack of video-enabled phones and inconsistent mobile phone ownership among ASHAs.)
• ASHA refresher training on mothers' group meeting logistics (e.g., operating the video projector) and facilitation skills.

A community advisory board (CAB) select topics for the videos on maternal, newborn, and child health, and board members with technical expertise provide key messages. In this iteration of the project, 6 videos of a total of 12 focused on immunisation topics. Community members from the intervention villages then created the storylines and acted out the key messages; some efforts were made to directly involve known sub-populations, such as Muslim community members. Performances were recorded using hand-held camcorders, and video production occurred within a short time frame: approximately one month from storyboard to screening. Between February 2017 and January 2018, ASHAs conducted monthly video screening sessions, totaling 1,558, with 25,343 video views. Following all screenings, ASHAs facilitated discussion based on the videos, answered questions, and reinforced key messages.

The theory of change (TOC) hypothesised that the intervention components would effect changes at three levels: mothers' attitudes and beliefs, mothers' networks, and community attitudes and social norms related to immunisation. The TOC rested on the following contextually driven assumptions:

• Healthcare-seeking behaviour change in rural India is highly dependent on family- and community-level attitudes and norms.
• Awareness of where and when to access services and care seeking behaviour is largely driven by ASHA outreach.
• During the government-mandated VHNDs, vaccines are available, ASHAs are able to mobilise families, and well-trained frontline health workers are available and able to effectively and safely administer vaccines and provide education and counselling.

For the cluster-randomised, controlled trial, baseline data collection was done from November 3 to November 30 2016, and endline survey data collection was done from September 1 to September 21 2018. A single cluster represented a single ASHA and the villages and hamlets she is responsible for serving. Seventy-four ASHA-village clusters within Airaya and Hathgaon blocks were pseudo-randomly selected. Half of these clusters were pseudo-randomly assigned to receive the Projecting Health intervention, and the remaining half acted as controls. The study participants, all of whom resided in the 74 selected study clusters, were: women 18 to 45 years old with children between the ages of 6 and 17 months; the women's husbands and mothers-in-law; and ASHAs.

To gather qualitative data, the team conducted interviews and focus group discussions (FGDs) from January 2017 to June 2018, with 373 participant, as well as 12 observations of VHNDs.

Main findings:

• While overall attrition in the intervention was low, the rate of first-time viewers quickly declined after the first three months. Also, some families were never enrolled due to factors related to household socioeconomic status, cultural and community norms, and ASHA behaviours and practices. For instance, ASHAs encountered families who actively avoided video screenings for cultural reasons.
• ASHAs were not trained in the different types of participants or how to identify and mobilise women using their knowledge of eligible mothers. Though male household members and mothers- in-law are influential in decisions to vaccinate, they were also under-mobilised; the video messaging was not specifically created or targeted to them. ("Future iterations of this intervention should tailor the intervention and its implementation strategies based on formative research and ongoing rapid cycle testing to maximize reach, particularly among the most vulnerable and household decision-makers.")
• The intervention did not have a significant impact on the full immunisation rates of infants. In the intervention areas, there was a non-significant 2.8 percentage point increase in the probability that an infant was fully immunised compared to the control clusters. (Note: For seven months during the study period, activities by the Government of India's Intensified Mission Indradhanush (IMI), an initiative to ensure full immunisation of all children in India, were implemented across all intervention and control villages. The majority of observed improvements in vaccination coverage in intervention clusters (51.9% to 64.7%) are likely attributable to IMI, with additional marginal improvement (3.9%) from the intervention.)
• The effect on timeliness of vaccination among children was marginal, with a 0.8 percentage point increase in the probability that a child in the intervention arm received all vaccines on time. The probability that a child received the first but not the third dose of diphtheria, pertussis, and tetanus (DPT) decreased by 6.78 percentage points in intervention villages relative to control villages. While all outcome indicators changed, their magnitude was not statistically significant.
• The intention to vaccinate was very high for mothers at baseline (97.7%) and endline (98.6%). Comparing intention between intervention cluster respondents who reported viewing an immunisation video at endline versus those who did not illustrates that viewing videos was associated with higher intent in the intervention clusters.
• There were individuals who actively chose not to vaccinate their children, which may reflect family constraints or logistical barriers that prevent a mother from making a vaccine decision, regardless of her own personal intention to vaccinate.
• Viewing a video was associated with greater knowledge of antenatal care, birth preparedness, breast feeding, and immunisation.
• Among the treated population, women who saw an immunisation video reported a different composition of social ties at endline compared to baseline and control clusters, with a greater proportion of these women's ties going to ASHAs: The intervention seemed to have a positive effect on the ASHAs' engagement with community members, even outside the video screenings.
• Exploratory analysis suggests that certain subgroups such as families in communities identified as HTR have benefitted from the intervention. In part due to their geographic location, ASHAs were often previously unlikely to visit these families as part of their outreach and mobilisation efforts. The intervention increased the likelihood that ASHAs engaged with these families, including through targeted video screenings with smaller groups of HTR women in their homes.
• The process evaluation suggested there was a difference between modifiable beliefs (such as fear of side effects) and deeply held beliefs that are less mutable (such as the belief among some Muslim families that vaccines cause infertility). Despite efforts to include Muslim actors in the videos and address deeply held beliefs, the intervention was not successful in increasing vaccination in this subgroup.

The authors describe several challenges they encountered during both implementation (e.g., societal norms around gender introduced constraints to men's screenings and screenings for newly married women) and evaluation (e.g., incomplete or missing immunisation cards during data collection posed a concern for calculating key outcomes pertaining to timeliness).

The authors propose possible reasons for the null findings, including key supply-side constraints that the intervention was not designed to mitigate. For instance, there was suboptimal quality among the auxiliary nurse midwives (ANMs) who deliver routine vaccines at VHNDs, which can lead to lack of trust among caregivers. Furthermore, the process evaluation uncovered additional drivers of vaccine coverage that were likely underestimated in the original TOC, such as the importance of family members other than the parents in vaccine decision-making. For instance, the deeply held belief related to vaccines and infertility remains a small but significant barrier that was largely unchanged by this low-intensity intervention. Results of in-depth probing during FGDs and ASHA interviews suggest this belief continues to persist in some sub-communities in this district and is more strongly held among older generations, who tend to be decision-makers in traditional households, as well as in Muslim communities.

In short: "To optimize the impact of this intervention, it may have been more effective to focus on attitudes and beliefs that were possible to change, particularly among key decision-makers. Localized knowledge of the social context and beliefs is critical to design messaging that specifically targets modifiable beliefs, which Projecting Health did not identify during program design. Future iterations should conduct initial rapid testing cycles to identify local beliefs and effective messaging." Other recommendations include:

• Policymakers and partners who design interventions to increase vaccination coverage should consider how these interventions can more effectively reach other family members within the household who serve as decision-makers (e.g., male household heads, elder women).
• The Fatehpur health district should adopt and implement best practices for refresher training and in-service training of ANMs based on identified competency gaps.
• State and district officials should identify barriers to ANM hiring and retention and address them through carefully implemented Human Resources for Health strategies.
• Districts should improve the design and implementation of ASHA training and supervision to ensure effective and consistent outreach to all families within their respective catchment areas.
• District-level stakeholders should emphasise and strengthen ASHA engagement and outreach through improved motivation and supportive supervision. Such strategies should provide financial or nonfinancial incentives for reaching vulnerable or under-immunised children and supportive supervision for tools such as due lists.
• Funders should consider funding novel evaluation approaches that enable continuous learning and improvement.

The authors conclude that there is a need for further research on the importance of high-quality clinical care to engender trust at the community level, engagement of all household members in the vaccine decision-making process, and continued and expanded outreach to vulnerable communities.

Editor's note: A finalised version of this report was published in August 2020; click here to view or download it (93 pages).