Measuring Vaccine Hesitancy: Field Testing the WHO SAGE Working Group on Vaccine Hesitancy Survey Tool in Guatemala

"[T]here is a need for a broader, validated, and reliable tool to measure the changing prevalence of vaccine hesitancy in LMICs that explores the complex factors affecting hesitancy."

Vaccine hesitancy occurs on a continuum between those who undoubtedly accept all vaccines to those who undoubtedly refuse all vaccines. Understanding the complex and interplaying factors that influence vaccination decisions and the determinants of vaccine hesitancy in a specific population can help inform interventions that seek to improve vaccine coverage. However, vaccine-hesitant parents represent a heterogeneous group, making the development of a survey tool difficult. To that end, the World Health Organization (WHO) Strategic Advisory Group of Experts (SAGE) Working Group on Vaccine Hesitancy developed a common diagnostic tool, the Vaccine Hesitancy Scale (VHS), to identify and compare hesitancy in different global settings. This study investigates the reliability and validity of the VHS measure applied in urban and rural Guatemala using a factor analysis. The objective is to provide insight into the shared understanding of the VHS construct using the tool in diverse global settings, including low- and middle-income countries (LMICs) like Guatemala.

In the context of a larger randomised intervention study exploring the use of short messaging service (SMS) technology to remind families about childhood immunisations, the study team conducted baseline assessments that incorporated 10 dichotomous (yes/no) and 10 Likert scale (strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree) VHS questions. The cross-sectional data were collected between March to November 2016 at 4 public health clinics of the Ministry of Public Health and Social Assistance in Guatemala, all of which serve a low-income population. Eligible participants included parents of infants between the ages of 6 weeks to 6 months presenting for their first wellness visit.

Table 2 in the paper provides parental responses to the closed-ended (Q1-Q10) and Likert scale (L1-L10) VHS questions. In short, of the 720 participating families, no parent had ever refused a vaccination, and only 8 parents (1.1%) had been reluctant or hesitated to get a vaccination for their children. No parent could think of a reason why children should not be vaccinated, and only 3 parents (0.4%) did not believe that vaccines could protect children from serious diseases. However, a majority of those interviewed (n = 426, 59.2%) thought that parents like them do not have their children vaccinated with all the recommended vaccines, with more urban versus rural parents expressing this view (69.7% vs. 48.6%; p < 0.0001). Time, distance, and cost to get to the clinic and/or timing of the clinic and wait at the clinic were significant factors thought to prevent immunisation more in the urban compared to rural population. While 205 (56.9%) parents in the urban clinics compared to 4 (1.1%) parents in the rural clinics believed that it was more difficult for children from some ethnic or religious groups in their community to get vaccinated (p < 0.0001), more rural than urban parents thought that their local leaders (religious or political, teachers, health care workers) did not support childhood vaccinations (45.6% vs. 18.6%; p < 0.0001). These findings highlight the importance of socio-cultural influences on vaccine attitudes.

The researchers explain that in Guatemala, "there are important structural issues to consider that may contribute more to limited vaccine compliance than hesitancy. For example, the political instability that occurred during our study led to considerable country-wide vaccine shortages and likely played a role in vaccine timeliness and compliance, which may have impacted parental attitudes and perceptions around immunization. We believe it is critical to consider how shortages, a reality experienced by many LMICs, affect vaccine timing and completion and, subsequently, may impact parents' vaccine hesitancy survey responses. We further posit that a failure to do so risks conflating attitudes towards vaccines with compliance, instead of making attitudes distinct from access issues..."

While the overall study population had very favourable attitudes towards vaccination, there were notable differences in the Likert scale level of agreement between clinics. Factor analysis identified 2 underlying constructs that had eigenvalues of 1.0 or greater and a substantive lack of variability in response across the Likert scale. The results "suggest difficulty understanding and using the Likert scale format." The researchers had very few neutral responses of "neither agree nor disagree". They also saw a consistent difference between how study clinics responded to the ordinal scaling, with one clinic almost always responding "agree" while the other 3 clinics almost always responded "strongly agree". This pattern seems to imply a user-bias in how the study nurses may have asked the questions and interpreted and recorded the participant's answers. Research has previously suggested that Likert-type scale formats may be culturally biased; for some populations, a graded response format measuring multiple degrees of variation may be too abstract and, in fact, meaningless. It has been suggested that a Likert-type scale format may be better understood and utilised amongst populations with a higher level of underlying education.

These results indicate problems with interpretation of the VHS, especially in the presence of vaccine shortages, as were experienced by all of 4 clinics, and using a Likert scale that does not resonate across diverse cultural settings. Furthermore, the actor analysis findings suggest that the Likert scale items collectively are more one-dimensional, primarily measuring the concept of confidence, and do not correlate well with the other constructs of complacency and convenience included in the definition of vaccine hesitancy. The researchers argue that future modification of the VHS would benefit from including multiple dimensions of vaccine hesitancy, especially related to the concept of convenience, with both positively and negatively worded items. Thus, the researchers conclude that more work is needed to refine this survey for improved reliability and validity.