Effectiveness of a Mobile Short-Message-Service–Based Disease Outbreak Alert System in Kenya

We conducted a randomized, controlled trial to test the effectiveness of a text-messaging system used for notification of disease outbreaks in Kenya. Health facilities that used the system had more timely notifications than those that did not (19.2% vs. 2.6%), indicating that technology can enhance disease surveillance in resource-limited settings.


Intervention
The intervention was a mobile SMS-based disease outbreak alert system (mSOS), which was developed by the Ministry of Health (MOH) in collaboration with the Faculty of Information and Technology at Strathmore University in Nairobi and was pretested and refined at several health facilities before training and deployment began in the study areas.
The mSOS consisted of formatted SMS communication between health workers at local facilities and MOH managers at the subcounty, county, and national levels. A web-based mSOS portal was developed and used to monitor notifications sent by health facility workers and response actions taken by the national disease surveillance officers and managers ( Figure   1 in main text). Health workers used an mSOS text messaging system for 6 months to send patient-level information for suspected cases that required immediate (i.e., within 24 hours) notification. Twelve diseases and conditions listed in the national IDSR guidelines were selected for the study (Technical Appendix Table). Text messages sent by health workers consisted of prescribed codes specifying patients' disease diagnosis, age, sex, and survival status (i.e., alive or dead). The messages were sent to a toll-free number set up by a telecommunication provider in Kenya. Health managers at all levels received text messages in real time on their mobile phones. By using a password-protected web-based portal, they could also observe all notifications, maps with locations of health facilities where incidents occurred, and graphs showing cumulative cases reported. All information sent by mSOS was stored on a secure server at the MOH.
Before mSOS was implemented in the study areas, a 1-day IDSR refresher training for all in-charges of health facilities was conducted during September and October 2013. The training focused on case definitions of notifiable diseases and routine paper-based case notifications. During the training, health facilities were randomized into control or intervention groups, and participants from the intervention group facilities received an additional day of training on using mSOS. During the mSOS training for health workers, the subcounty and county disease surveillance coordinators were also trained on how to access and use the web-based portal to view mSOS information and how to log the response actions taken. Throughout the study period, the paper-based reporting, as indicated in the national IDSR guidelines, continued in both the intervention and control facilities; the intervention group was also trained to use mSOS to report the same cases reported by paper.

Randomization and Masking
Randomization was conducted during the IDSR training by stratifying health facilities by subcounties and randomly selecting intervention facilities from each stratum by using a 1:1 ratio. The intervention group was unmasked because of the nature of the study; investigators, health workers, and health mangers were observing SMS notifications and were aware of which study facilities were in the intervention group.

Data Collection
To evaluate the intervention, pre-and post-intervention surveys were undertaken by the health facilities. In June 2013, baseline retrospective data were collected for the 6-month period before the intervention (December 2012-May 2013). In May 2014, the data were collected for the six-month duration after the intervention was launched (November 2013-April 2014). At each study facility, trained data collectors reviewed all outpatient, inpatient, and maternal and child health registers and extracted patient-level information for the 12 diseases and conditions selected for the study. For each extracted case, date of patient's visit, name, sex, age, and provisional diagnosis were recorded. Copies of submitted paper-based reports for immediately notifiable diseases were also reviewed. In addition, the visitors' and supervision books signed by surveillance coordinators were reviewed at the health facility to determine whether any response action was taken at the health facility after the notification was sent.
For the intervention group, notifications sent through mSOS were also extracted.
During the surveys, all in-charges of health facilities were interviewed, and information on characteristics of the facility and of the in-charges managing the facility and their exposure to the intervention was recorded. Data extracted from facility records were collected on paper forms, and data from structured interviews with facility in-charges were collected by using Magpi software (5) installed on data collectors' mobile phones.

Definitions
A case requiring immediate notification was defined as any of 12 notifiable diseases and conditions extracted from any of the facility registers. Data from the source documents were examined to eliminate duplicate cases by using patient's diagnosis, date, and name.
Notifications were defined as cases reported through paper-based forms in the control group and mSOS or paper forms in the intervention group. A response action taken was defined as visits to the reporting facility by the subcounty, county, or national surveillance coordinators, as documented in visitors' or supervision books in the control group or through the mSOS web portal or visitors' or supervision books in the intervention group. According to the national IDSR guidelines in Kenya, the 12 notifiable diseases and conditions, except for measles, required immediate notification within 24 hours of detection, and response action was required within 24 hours of notification. Measles required immediate notification within 24 hours of detection, and response action was required within 24 hours of notification of the fifth suspected measles case detected in the same health facility or subcounty during 1 month.
The number of notification days was calculated as the period of days between the date of case detection and the date of notification.

Statistical Analysis
All analyses were performed by using Stata version 12 (College Station, Texas, USA; http://www.stata.com/). The primary analysis was "intention-to-treat" and included cases from all study facilities as the facilities were randomized, regardless of the intervention exposure. The secondary analysis was "per-protocol" (i.e., study protocol) and was restricted to the cases from the facilities where the facility in-charges were exposed to the IDSR training in the control group and to IDSR and mSOS training in the intervention group. An additional analysis in which per-protocol conditions were further restricted to the casepatients seen at health facilities with available paper-based tools was also performed. To explore potential confounders, the χ 2 test for proportions and the Wilcoxon Mann Whitney test for median comparisons were conducted on characteristics of health facilities and of their in-charges to compare the control and intervention groups. Because no significant differences were found and only 1 notified case across study groups was reported at baseline, results from the post-intervention survey under the intention-to-treat and per-protocol analyses were the primary focus of results (presented in this article. Because of the small population sizes for both analyses, which precluded cluster adjustments, we calculated the 95% CIs around differences in proportions between notification outcomes for the intervention and control groups by using the Wilson procedure with continuity correction (6,7). CI estimations were done at an α level of 0.05. were also found to be contaminated with in-charges who were exposed to the SMS component of the intervention. The restricted per-protocol analysis included 64 health facilities (32 in the intervention group and 32 in the control group).

Limitations
This study has several possible limitations. First, we did not capture possible informal notifications through phone calls or in-person interactions between health facility in-charges and disease surveillance coordinators at the subcounty, county, and national levels. This lack of information may have underestimated the true rates of managers' awareness about immediately notifiable cases, but the information collected in the study reflects the true degree of compliance with the IDSR national guidelines. Second, the lack of completeness of notifiable cases recorded by health workers in the source documents at the health facilities is an inherent problem and may have resulted in selection bias. Such bias was partly remedied through the randomized design. Finally, the study did not verify whether the diseases recorded in the source documents were correctly diagnosed or recorded on the basis of case definitions or laboratory confirmations; these verification measures were beyond the scope of the study.