Digital Health for Real-Time Monitoring of a National Immunisation Campaign in Indonesia: A Large-Scale Effectiveness Evaluation

"While the campaign experienced challenges related to vaccine hesitancy, strong associations were observed between platform use and the achievement of coverage targets...suggesting that feedback from the platform stimulated corrective action to help districts overcome challenges."

Indonesia has committed to eliminating measles and controlling rubella and congenital rubella syndrome (CRS) by 2023. To achieve this aim, the country undertook a nationwide measles and rubella (MR) vaccination campaign in 2017/2018. This paper assesses the contribution of a digital health real-time monitoring platform to the achievement of immunisation coverage targets. The platform was implemented during the campaign's second phrase, which was designed to reach 31.9 million children between 9 months and 15 years old throughout the Indonesian archipelago, including socioeconomically and geographically diverse provinces with varying levels of infrastructure, capacity, and connectivity.

The digital health platform was developed using the United Nations Children's Fund (UNICEF)-designed open source software RapidPro, which was administered at no cost to users. It was intended to be complementary to paper-based reporting, and not a substitute. Immunisation managers and health facility staff were trained in the use of the tool, including through an instructional animated video disseminated through WhatsApp. The trained vaccinators then used personal mobile phones ("smart" or simple) to register their health facility; at the end of each reporting day, they then submitted a text message (SMS) to report the number of children immunised. Data from all facilities were aggregated and visualised with an online dashboard at the district, provincial, and national levels. Vaccinators and programme managers received an automated text message at the end of each day with feedback on progress towards coverage targets for their administrative level.

Data collected during the campaign on platform use and study outcomes were analysed at the district level comparing varying degrees of implementation. The evaluation made use of both quantitative and qualitative data collected before, during, and after the campaign, and it accounted for baseline performance of district immunisation programmes.

In brief, between August 1 and December 31 2018, 6,462 health facilities registered to use the digital health platform across 28 provinces and 395 districts. A total of 72.7% of facilities reported at least weekly into the system. Among the reasons for irregular or non-reporting were, for example: mobile connectivity challenges, and facilities that stopped participating in the campaign due to demand-related delays in vaccine acceptance. After adjusting for precampaign district risk profile and intracampaign delays due to vaccine hesitancy, districts with greater platform utilisation demonstrated higher coverage overall (R2=0.28, p<0.0001) and a shorter interval to achieving full coverage. Stronger effects were observed among districts experiencing implementation delays due to vaccine hesitancy.

Furthermore, results from 106 key informant interviews conducted face to face in 6 provinces and 18 districts suggest high degrees of acceptability, ease of use, and satisfaction. For instance: "The timeliness and ease of use of the platform were preferred over paper-based reporting and phone calls used in past immunisation campaigns and programmes. Health facility informants appreciated the daily reminders, which they said improved their motivation to report, creating a sense that their work was recognised and appreciated." Furthermore: "Both health facility vaccinators and programme managers highlighted that daily RapidPro updates on immunisation coverage triggered discussions through established social media networks....This was reinforced by the online data visualisation platform, which fostered greater accountability within and between districts."

Factors the researchers cite as having contributed to the adoption and effective use of the platform include:

• High-level endorsement by the Ministry of Health, which was informed by positive experience with RapidPro during 2017 across a more limited geographical area;
• Real-time data that enhanced local coordination, with nearly three-quarters of respondents reporting that the platform facilitated problem identification and advocacy, particularly among districts affected by vaccine hesitancy; and
• Indonesia's high level of digital connectivity, with at least 72% of the population using mobile phones and 56% connected to the internet and social media.

Although issues of vaccine hesitancy were overcome in most instances with the help of RapidPro, this was not the case in several districts - "highlighting the importance of parallel accountability and problem-solving mechanisms."

In conclusion, the researchers suggest that RapidPro's application "across a diverse range of contexts with varying range of data literacy and connectivity underscores its generalisability within the country and elsewhere. Future applications of this approach to other campaigns, to surveillance activities and to routine programming are ongoing areas of work. Finally, digital innovations that combine supply-side decision support for health workers with efforts to improve demand-side awareness among programme beneficiaries are important areas to pursue in future programmes and research."