Using Preference Parameter Estimates to Optimise Public Sector Wage Contracts

"Using smartphones to collect data on the performance of polio vaccination campaign workers is a cheap and reliable way to improve the existing monitoring activities. The use of geo stamped and time stamped data provides a good picture of the extent of vaccination coverage and can easily identify blind spots in paper-based administrative divisions. It has also been useful in identifying vaccinator performance in an objectively verifiable manner."

This research report explores an information and communication technology (ICT)-based approach to one barrier to full polio immunisation coverage in Pakistan: low health worker motivation. With funding by the International Growth Centre (IGC), the researchers introduced a set of technological and incentive contract innovations aimed at understanding health worker behaviour and ultimately making door-to-door polio vaccination efforts more effective in Lahore District. As part of these door-to-door campaigns, teams of 2 vaccinators, typically a government worker and a volunteer from the community or from another department of the government, go door to door providing oral polio drops. The makeup of the team fills these needs: helping government outreach workers engage with the community and creating ownership from within the community through the engagement of volunteers.

Prior to the intervention, the performance of polio vaccinators was costly to measure. The status quo procedure was to provide vaccinators with a map of potential households and ask them to self-report the vaccinations completed on each day of a multi-day drive. In this system, 4 or 5 roving teams of health workers work to vaccinate a neighbourhood. They use paper maps, record their efforts on paper forms, and employ a protocol of writing in chalk on the walls of houses to indicate to other teams that they have been there. For an independent monitor to check up on a team's efforts, she would have to physically follow their footpath through the neighbourhood, reading the chalk marks. As a result, health workers often fail to achieve their targets, but report that they have.

So, in collaboration with the Department of Health, the researchers designed a smartphone-based monitoring system called Polio survey that allowed them to track the door-to-door vaccination activities by geo-stamping and time-stamping the vaccination attempts. Data from the smartphone system are aggregated in real-time on a dashboard available to senior health administrators so that they can observe the extent of vaccination coverage. The smartphone system also allows the implementation of pay-for-performance contracts, because vaccinators' efforts can be observed precisely. For a 2-day vaccination drive, the Health Department required workers to complete a given number of vaccinations (300) in exchange for a fixed bonus (US$10). Vaccinators could allocate vaccinations either to the first day or to the second day of the drive, according to their preferences. In a between-subject design, the researchers assigned vaccinators different "interest rates", or the rate at which one could trade vaccinations on day 1 of the drive for vaccinations on day 2 of the drive. Additionally, between subjects, some vaccinators made their allocations 3 days prior to the vaccination drive, while some made their allocations on the first day of the vaccination drive. This design permits the identification of time preferences and dynamic inconsistency for the sample of vaccinators. The researchers found that 55% of vaccinators prefer to delay tasks more when they are making an immediate choice than when they are making an advance choice based on their decisions from consecutive drives. The between-subject tests also provide substantial evidence of dynamic inconsistency.

In Lahore's Allama Iqbal and Nishtar Town areas, 505 LHWs participated in the trial. For the research, they were given smartphones with the app and instructed to set targets of the number of houses they wanted to visit across 2 days. During the trial, they fed into the app information about the number of children in each household they visited. The information, including Global Positioning System (GPS) data on where they were and what time they visited each household, was received by the researchers in real time.

As part of a pay-for-performance scheme, one group of LHWs (336 out of the 505) could receive bonus payment in addition to the flat rate, if they met their target number of households for both days. The other group (85 workers) only received a flat rate, and the remaining 85 were treated as a pure control group. The result: Smartphone data showed that LHWs working with the possibility of a bonus administered 15-17% more vaccinations than LHWs with only a flat rate. Analysing data from this trial, researchers came to a conclusion that would form the basis for the next incentive: LHWs often did not meet the targets they set for themselves. During a 2-day campaign, some LHWs were too ambitious on the first day and fell short. Others put off too much work to the second day, and found themselves rushing to meet their targets. Rushed service, the researchers speculated, was not smooth service. In a typical campaign where workers tried to meet the target of around 300 houses per day, polio drops could be administered incorrectly, and less effort could be made to convince hesitant parents to vaccinate their children. To address this problem, a new scheme was introduced which tailored vaccination targets according to each LHW's behaviour in the first trial. Depending on how a LHW set targets the first time around, the scheme imposed a penalty that either added or subtracted from the value of one visit towards the total goal of 300. The researchers found that tailoring vaccinations according to personal preferences increased the likelihood of adopting a more balanced schedule by 33%. This self-tailoring incentive structure succeeded in evening out efforts: the rise in vaccinations was achieved by spreading the effort over the 2 days of the drive. Furthermore, in the few cases where parents didn't allow their children to be vaccinated, the online dashboard allowed policymakers to identify those households for follow-up.

Health workers report that this technology may also help make them appear more credible to parents of children. "Using smartphones in front of parents legitimises our presence," says Chand Sultana, a LHW who participated in the trial.

There are limitations to these results. Firstly, replicating these policies in districts with different political, social, and economic contexts may not yield the same results. Another problem was that LHWs were unaware that their tailored targets for the second trial were based on their behaviour from the first trial. If they knew this, there is a possibility they would alter their behaviour to game the system. But the researchers say that as long as incentives continue to be adjusted based on the most recent vaccination campaign data, this should not be a problem.

The researchers have recommended to the government that adopting smartphones for monitoring LHWs could offer a viable solution for increasing vaccination coverage. "Through this experiment we have shown that such a scheme is possible for front line public health workers and can lead to significant improvement in performance at minimum cost."