Determinants of Facility-Level Use of Electronic Immunization Registries in Tanzania and Zambia: An Observational Analysis

"New systems that aim to improve data timeliness, availability, or completeness will only be able to do so if they are used consistently as intended."

Traditionally, most low- and middle-income countries (LMICs) have relied on paper-based data collection at the facility level to capture immunisation data. The electronic immunisation registry (EIR) has been proposed as a means of improving data quality, facilitating reporting, and promoting data use in LMICs, ultimately providing the opportunity to strengthen vaccination services. This observational study examines the organisational, technical, and behavioural factors associated with facility EIR use in Tanzania and Zambia between 2016 and 2018.

As detailed in the paper (with more information at Related Summaries, below), Tanzania and Zambia are demonstration countries in the Better Immunization Data (BID) Initiative. Funded by the Bill and Melinda Gates Foundation, the BID Initiative included both technological and change management components to foster an environment conducive to data use for decisionmaking. The interventions were introduced to health care workers (HCWs) in each country through on-the-job training with staff from higher levels of the health system, who were engaged to provide a supportive environment by championing data use practices, mentoring facility staff, and holding facilities accountable for their performance.

The present study used EIR data entered between 2016 and 2018 in 3 regions of Tanzania (Arusha, Kilimanjaro, and Tanga) and in Southern Province, Zambia to measure weekly EIR system use for 50,639 facility-weeks. They also drew on secondary data on facility characteristics and applied the Performance of Routine Information System Management framework to categorise characteristics as:

• Organisational: e.g., level of supervisory and political support for the new system, availability of human and financial resources, and management support. These factors were manifested at the facility, district, or regional level through informal norms, values, and practices or through formal guidelines, standards, and policies.
• Technical: e.g., user-interface design and offline functionality. These factors were expected to affect the user's experience with an EIR system as well as the system's feasibility and acceptability.
• Behavioural: e.g., HCWs' capability and motivation to use the new system.

The researchers used a generalised estimating equations logistic regression model to assess facility characteristics as potential determinants of system use. Selected findings:

• In both countries, the estimated odds of weekly EIR use declined weekly after EIR introduction.
• In Tanzania, 2 organisational determinants - facility type and whether the facility had transitioned fully to paperless reporting - were significant predictors of EIR use. Specifically: (i) compared to dispensaries, health centres were 61% more likely to use the system, while hospital odds of use were 3.83 times greater; and (ii) facilities that had transitioned to completely paperless reporting had odds of weekly EIR use that were 2.76 times as large as facilities using parallel EIR and paper reporting systems. (The researchers surmise that HCWs may have lost motivation to continue using the system if they perceived it as adding more work, since they were still required to use the paper-based system for official reporting.) For each additional HCW that was trained during the EIR introduction, estimated odds of weekly EIR use were 1.39 times greater.
• In Zambia, distance from the district health office (DHO) was significantly associated with decreasing odds of system use. One hypothesis: More remote facilities may have been less likely to have received supportive supervision. Distance from the DHO may have also captured farther facilities having limited infrastructure to support the technology, therefore lower likelihood of EIR use.

In short, the technical covariates tested were not statistically significantly associated with weekly EIR use in the final model for either country or were excluded due to missing data. Thus, the results highlight the importance of organisational and behavioural factors in explaining sustained EIR use. For instance, individual or facility recognition or incentives could have been powerful motivators to support behaviour change but were applied inconsistently; failure to recognise consistent use combined with a lack of perceived data use may have contributed to waning motivation. "Across all facilities, additional support and accountability to encourage use of the EIR was needed to sustain use over time."

As more countries move to introduce EIRs or other digital interventions, the researchers recommend that indicators of engagement and use be built directly into the system so they can be routinely monitored, and course corrections can be implemented as needed. Other factors that may contribute to successful uptake and ongoing use of these systems:

• An emphasis on engagement from local leaders at the subnational level;
• Strong district leadership and the mentorship and close supervision of HCWs;
• Continued efforts to: transition facilities to paperless reporting, promote the benefits of system use to HCWs, and make system use metrics available (e.g., through automated reports or dashboards) to empower stakeholders at all levels; and
• Additional support to more remote, lower-volume facilities.