Volume 22, Number 3—March 2016
Faster Detection of Poliomyelitis Outbreaks to Support Polio Eradication
As the global eradication of poliomyelitis approaches the final stages, prompt detection of new outbreaks is critical to enable a fast and effective outbreak response. Surveillance relies on reporting of acute flaccid paralysis (AFP) cases and laboratory confirmation through isolation of poliovirus from stool. However, delayed sample collection and testing can delay outbreak detection. We investigated whether weekly testing for clusters of AFP by location and time, using the Kulldorff scan statistic, could provide an early warning for outbreaks in 20 countries. A mixed-effects regression model was used to predict background rates of nonpolio AFP at the district level. In Tajikistan and Congo, testing for AFP clusters would have resulted in an outbreak warning 39 and 11 days, respectively, before official confirmation of large outbreaks. This method has relatively high specificity and could be integrated into the current polio information system to support rapid outbreak response activities.
The global eradication of polio is entering its final stages. The last case of poliomyelitis associated with serotype 2 wild poliovirus was reported in 1999 and of serotype 3 in 2012. In Africa, the last reported case of serotype 1 wild poliovirus was in Somalia in August 2014. Transmission of this serotype has yet to be interrupted in Afghanistan and Pakistan, and in 2014, 359 serotype 1–associated cases were reported worldwide, 81% of which occurred in Pakistan (1).
Transmission of wild poliovirus persists in countries where the disease is endemic, but outbreaks can also occur in previously polio-free populations in which population immunity is not sustained. For example, the 2013 polio outbreak in the Middle East was linked to importation of poliovirus from Pakistan (2). The live-attenuated oral poliovirus vaccine (OPV) has played a huge role in achieving >99% reduction in global annual incidence of poliomyelitis, but its continued use also means there is a risk for emergence and spread of circulating vaccine-derived poliovirus (cVDPV) (3). In 2015, cVDPV outbreaks were reported in at least 5 countries (1). The risk for serotype 2 cVDPV may be heightened during the planned global switch from trivalent to bivalent (containing Sabin virus types 1 and 3) OPV during routine vaccination in April 2016 (4). Poliomyelitis outbreaks substantially raise the cost of the eradication program and hinder progress toward eradication, particularly if they are not swiftly controlled (5). Early detection is therefore critical to the program to enable a fast outbreak response to quickly stop transmission.
Surveillance for poliomyelitis relies on the reporting of cases of acute flaccid paralysis (AFP) in children <15 years of age by healthcare providers (Figure 1 at http://dx.doi.org/10.5281/zenodo.44361) (4,6). In some areas this surveillance is supplemented by environmental surveillance, which involves the periodic collection and testing of sewage samples for the presence of polioviruses. Surveillance is challenging because of the large number of asymptomatic cases (100–1,000 infections/AFP case) and because there are multiple causes of AFP (e.g., trauma, toxins, enteroviruses), thus requiring laboratory testing of stool samples to confirm the presence of poliovirus (7–9).
In 2010, large outbreaks of poliomyelitis in Tajikistan and Republic of the Congo (Congo) were detected relatively late, partly due to delays in laboratory processing of stool samples; the delayed detection resulted in a limited effect from the outbreak response vaccination campaigns (10). The high transmissibility and pathogenicity of wild and vaccine-derived polioviruses means that poliomyelitis cases may be expected to cluster in space and time to a greater extent than do cases of AFP associated with other enteroviruses or noninfectious causes. We therefore decided to investigate whether clusters of AFP could herald poliomyelitis outbreaks and be identified as an early warning of outbreaks before laboratory confirmation.
Cases of AFP are reported through a network of healthcare providers as part of routine surveillance for poliomyelitis (6). We analyzed 67,218 AFP cases with clinical onset during 2003–2013; the patients resided in 3 countries that had >150 confirmed cases of polio annually since 2005 (Tajikistan, Congo, and Somalia) or in countries in Africa considered to be at high risk for an outbreak of wild poliovirus. For each AFP case, the following information was recorded: the first and second administrative levels (province and district, respectively) in which the patient resided; the dates of AFP onset, case notification, and stool sample collection; and the patient’s age and sex.
AFP cases with stool samples adequate for testing were distinguished as virologically confirmed cases of poliomyelitis caused by wild poliovirus type 1 or 3 (3,089 cases), cVDPV (70 cases), or nonpolio AFP. Cases of AFP in persons without adequate stool samples were defined as polio-compatible cases (i.e., cases with clinical symptoms compatible with poliomyelitis, as determined by a panel of experts; 1,436 cases) or nonpolio AFP cases. The total number of nonpolio AFP cases was 62,623.
Institutional ethics approval for this study was not sought because the databases are free of personally identifiable information. National and subnational (first administrative and second administrative levels, respectively) boundaries were obtained from the World Health Organization.
For each of the 20 countries in the study, we obtained raster population size data for 2010 from the WorldPop project (http://www.worldpop.org.uk) (11,12). The data contained estimates of population distribution at ≈100-m2 spatial resolution. Population size estimates at the district level were acquired by aggregating the raster data within each district by using the R package raster (13) (Figures 2 and 3 at http://dx.doi.org/10.5281/zenodo.44361) implemented in the R programming language (14).
Time from Paralysis Onset to Case Notification to Specimen Delivery for Laboratory Testing
Delays in reporting and testing were determined for all AFP cases reported from Africa during 2010–2013 with available date information. We computed the length of time between paralysis onset and case notification and between case notification and the date stool samples were sent to a global polio laboratory for testing.
Space–Time Analysis of Nonpolio AFP data
We fitted a mixed-effects spatiotemporal statistical model to the data for each country (http://dx.doi.org/10.5281/zenodo.44361). In brief, the number of nonpolio AFP cases reported in a district at a given time was assumed to follow a Poisson or negative binomial distribution. In accordance with the model of Besag, York, and Mollié (15), the linear predictor was based on spatially structured and spatially unstructured random effects, with an additional offset of population size, and a random walk over time to account for temporal trends in reporting. The models were fitted to the nonpolio AFP data for each country in a Bayesian framework by using INLA (integrated nested Laplace approximation) (16) implemented in the INLA package (17). We selected the most parsimonious model, according to the deviance information criterion (18), to determine whether the count data followed a Poisson or negative binomial distribution.
Creation of Real-Time AFP Databases
A record is not kept of when each AFP case enters the AFP database. To test whether clusters of AFP cases could be identified in advance of an outbreak, we created real-time AFP databases for each Monday during 2003–2013 by assuming cases entered the database on the date the case was notified by local healthcare providers. These real-time databases partly capture the delay between symptom onset and reporting of AFP cases (Figure 1 at http://dx.doi.org/10.5281/zenodo.44361) and are a best-case scenario of timely reporting. When the date of notification was missing, we used the date of investigation, first stool collection, or second stool collection (in that preferential order) as proxy for when the case entered the database.
Testing of Real-Time AFP Databases to Detect Polio Outbreaks
For each country, the prospective Kulldorff Poisson space–time scan statistic (19) was evaluated at weekly intervals from the real-time AFP database to identify clustering of AFP cases in space and time. In summary, for every district in a given country, space–time cylinders were created; the cylinders were centered on the centroid of the district, and each had a different radius (representing various distances from the centroid to other district centroids) and height (representing different time periods up to and including the current week of surveillance). Cases of AFP were included in a given sized cylinder if the onset date for the case was within the interval of the start and end dates of the cylinder and the radius passed through the centroid of their reporting district. The cylinder end date was always the date of the real-time database; the start date varied from 1 to 90 days before the end date (we assumed standard methods of poliomyelitis outbreak detection would have detected an outbreak >90 days after the date of paralysis onset of the first AFP case). The maximum radius of the cylinder was restricted to 500 km, a conservative distance given the observed spatial clustering of polio cases at the start of an outbreak (Figure 9 at http://dx.doi.org/10.5281/zenodo.44361). The radius did not extend outside a given country.
The number of AFP cases observed within each cylinder was summed, and the likelihood ratio function, defining how likely there is an elevated risk within the cylinder compared with outside the cylinder, was maximized across cylinders of all locations and sizes. The expected rate of AFP reporting in the absence of a polio outbreak was obtained from the spatiotemporal regression model. The cylinder with the maximum likelihood ratio corresponded to the identified cluster (http://dx.doi.org/10.5281/zenodo.44361). The p value of the cluster was determined by Monte Carlo hypothesis testing by simulating cases under the null hypothesis and comparing the rank of the maximum likelihood ratio of the data with the simulations (http://dx.doi.org/10.5281/zenodo.44361). A cluster of AFP cases was defined to trigger an alarm of a potential outbreak when p<0.05. The space–time scan statistic was evaluated by using SaTScan version 9.3 (20), which was called using the R programming language (14), and the computation was parallelized over a 16-core, high performance cluster. We also tested the ability of the space–time permutation scan statistic (21) as an alternative to the space–time Poisson scan statistic (http://dx.doi.org/10.5281/zenodo.44361) because the space–time permutation scan statistic only relies on case data.
An outbreak was classified as detected by the algorithm if a warning alarm was raised within the outbreak period and if the location of the alarm occurred in at least 1 district containing reported outbreak-associated polio cases. The algorithm was assessed in its ability to detect confirmed serotype 1 and 3 wild poliovirus and cVDPV outbreaks. An outbreak period was defined as the length of time that consecutive, type-specific cases occurred with dates of paralysis onset <6 months apart. The percentage of outbreaks that were correctly identified was recorded (sensitivity of algorithm). The time of the alarm was compared with the date the outbreak was officially confirmed. The date of confirmation was not available for the smaller outbreaks; therefore, it was not possible to evaluate the timeliness of these alarms, apart from observing the time between the alarm and the date of onset of the first case. The specificity of the cluster detection algorithm was evaluated at the country level as the percentage of outbreak-free weeks without a false alarm. Sensitivity to this definition was examined (http://dx.doi.org/10.5281/zenodo.44361).
Nonpolio AFP Reporting
The number of nonpolio AFP cases reported at the district level was spatially heterogeneous within each country investigated throughout 2003–2013 (Figure 1; Figure 4 at http://dx.doi.org/10.5281/zenodo.44361). Spatial heterogeneity remained across all the Africa countries when adjusting for the population size of each district (Figure 5 at http://dx.doi.org/10.5281/zenodo.44361). The number of nonpolio AFP cases reported during 2003–2013 increased over time in all countries except Equatorial Guinea, although the rate of increase differed by country (Figure 6 at http://dx.doi.org/10.5281/zenodo.44361).
Time from Paralysis Onset to Case Notification to Specimen Delivery for Laboratory Testing
The median delay between onset of paralysis and AFP case notification was <1 week in all countries across the time period analyzed, although the distribution was skewed such that 5.9% (range 2.0% [South Sudan] to 10.7% [Côte D’Ivoire]) of AFP cases were notified >3 weeks after onset of paralysis (Figure 2; Figure 7 at http://dx.doi.org/10.5281/zenodo.44361). The median delay between AFP case notification and stool sample delivery to a global polio laboratory was <1 week across 2010–2013 in Central African Republic, Kenya, Madagascar, Cote D’Ivoire, Ethiopia, and Benin, and 1–2 weeks for the remaining countries (Figure 2; Figure 8 at http://dx.doi.org/10.5281/zenodo.44361). However the distributions were also skewed such that 3.9% of stool samples were dispatched to the laboratory >3 weeks after notification.
Spatiotemporal Model of AFP Reporting
Spatiotemporal mixed-effects modeling enabled characterization of the temporal trend and variability at the district level for each country through estimating the precision of the spatial and temporal random effects (Table 1 at http://dx.doi.org/10.5281/zenodo.44361). In all countries, there was evidence for at least 1 type of spatial random effect, indicating that the estimated district population sizes alone were not sufficient to explain differences in reporting rates. Evidence indicated overdispersion in the nonpolio reporting rate in 9 countries where a negative binomial model of nonpolio AFP case reporting provided a lower deviance information criterion value than that provided by a Poisson model. The country-specific model fits over time corresponded with the country data (Figure 6 at http://dx.doi.org/10.5281/zenodo.44361). It was possible to obtain the expected number of nonpolio AFP cases independent of time from these fitted models (Figure 1; http://dx.doi.org/10.5281/zenodo.44361).
Distribution of Poliomyelitis Cases and Nonpolio AFP Cases in Space and Time
Overall, compared with nonpolio AFP cases, poliomyelitis cases during the beginning of a large outbreak occurred closer together in space and time (Figure 9 at http://dx.doi.org/10.5281/zenodo.44361). In all large outbreaks during 2003–2013, most cases occurred within 100 km and 0–6 days of each other.
Testing of Real-Time AFP Databases to Detect Polio Outbreaks
Using the Poisson space–time scan statistic to test for the presence of AFP clusters at weekly intervals resulted in prompt warnings of a polio outbreak in the 4 recent large outbreaks (Figure 3). In Tajikistan, the detection of significant clustering would have occurred on March 15, 2010, which is 39 days before official confirmation of isolation of wild poliovirus on April 23, 2010. Clustering of AFP cases would also have been detected 11 days before official confirmation of the large outbreak in Congo (October 25, 2010, and November 4, 2010, respectively). However, in Somalia, 2 large outbreaks were detected by standard surveillance methods on August 23, 2005, and May 9, 2013, respectively, whereas the clustering algorithm would not have detected significant clusters of AFP cases until September 19, 2005, and May 20, 2013, respectively. The radius of each detected outbreak was 55.4 km (Tajikistan), 239.5 km (Congo), and 10.9 km and 21.1 km (Somalia); significance levels of the alarms were all p<0.001. In other settings, the algorithm detected some smaller outbreaks of polio (Table), although the time to detection was slow (Figure 10 at http://dx.doi.org/10.5281/zenodo.44361), and other outbreaks were not detected. In all countries, with the exception of the Democratic Republic of the Congo (DRC), relatively few false alarms were raised during outbreak-free periods (Table; Figure 10 and Table 4 at http://dx.doi.org/10.5281/zenodo.44361).
Overall, the space–time permutation scan statistic performed less well than the Poisson space–time scan statistic. The space–time permutation scan statistic would have resulted in a later detection of the 2010 Tajikistan and Congo outbreaks, and it detected fewer outbreaks in other countries (Tables 5 and 6 at http://dx.doi.org/10.5281/zenodo.44361).
Maintaining high-quality surveillance for polio outbreaks is essential to achieve global eradication of poliomyelitis. The longer the delay between the start of a polio outbreak and its detection (and subsequent response), the higher the chance of wide-scale spread and reestablished transmission. The large outbreak in Tajikistan in 2010 was detected relatively late (10), and during 2009–2010, outbreaks in Angola, Chad, DRC, and Sudan have led to reestablished transmission (5).
The duration of time between the onset of symptoms in the first reported polio case and confirmation of an outbreak can be prolonged due to delays in sending stool samples for laboratory testing and the time taken to perform the test. In addition, many countries do not consistently perform adequate stool collection to test for the presence for poliovirus (22). Our findings show that, compared with nonpolio AFP cases, poliomyelitis cases cluster in time and space, and that, in some instances, detection of spatiotemporal clustering of all-cause AFP cases can provide an early warning of outbreaks. Such a method has been shown to be an effective early-warning system for outbreaks of other infectious diseases (23–25). The method could be run on a weekly basis, as new AFP cases enter the database, and detection of a significant cluster would warrant fast-track laboratory processing of the stool samples from the associated AFP patients and alert countries to prepare for a possible outbreak.
By creating a real-time database, in which AFP cases were assumed to enter the database on the date of notification (best-case scenario of reporting), and running the spatiotemporal scan statistic at weekly intervals, an early warning of the large 2010 Tajikistan outbreak could have been raised 39 days before the date that the outbreak was officially confirmed. If outbreak response immunization campaigns had commenced 2–4 weeks earlier, substantially more poliomyelitis cases would have been prevented (10). In addition, an early-warning alarm of the 2010 Congo outbreak could have been raised 11 days before official confirmation. Therefore, incorporation of this early-warning system into the polio information system would benefit the Global Polio Eradication Initiative (GPEI). Although we found that the scan statistic would not have raised an early warning regarding the large 2005 and 2013 outbreaks in Somalia, the dates of the alarm were not long after the dates of official outbreak confirmation.
The algorithm performed less well at detecting much smaller outbreaks that have occurred during the past decade in countries of sub-Saharan Africa. During these outbreaks, the initial growth rate was relatively low compared with that in the outbreaks in Tajikistan, Congo, and Somalia, meaning there was little temporal clustering of polio cases. However, even if the sensitivity of early outbreak detection is not high for small outbreaks, the large outbreaks for which it does provide an early warning and hence a faster response will be of public health benefit, enabling more rapid outbreak control and a reduction in the number of poliomyelitis cases. The algorithm can be automated and, after future work to test the algorithm in other settings, would complement the current surveillance system.
Part of the polio endgame strategy is the globally synchronized removal of serotype 2 OPV from routine immunization in April 2016 (4). After this transition, there is a risk that cVDPV2 outbreaks will arise as population immunity against serotype 2 declines. Therefore, surveillance for cVDPV2 outbreaks will be critical during the transition period. Our results show that the algorithm we used would have generated alarms during cVDPV outbreaks in DRC, Cameroon, Kenya, and Somalia (Figure 10 at http://dx.doi.org/10.5281/zenodo.44361) and, thus, could be of help during the vaccine transition period.
A critical feature of an early-warning system is the false-alarm rate. A system with infrequent false alarms could benefit countries by providing a means to check the level of outbreak preparedness. However, a system that results in frequent false alarms is likely to be ignored when a true alarm is raised. In general, the false-alarm rate in our study was relatively low; an exception occurred in DRC, where false alarms would have been raised in late 2012–2013.
To obtain the expected proportion of AFP cases reported at the district level in the absence of a polio outbreak over a given time period, we fitted a spatiotemporal regression model to the incidence of reported nonpolio AFP during 2003–2013. We found that the number of nonpolio AFP cases reported per district was not simply a function of population size and that reporting is heterogeneous within countries. Subnational heterogeneous AFP reporting has been demonstrated at the first administrative level (province) in many settings (6,26,27), but fewer studies have investigated differences at the district level. Possible errors in population estimates, which could arise, in part, due to infrequent censuses, are a potential explanation for some of the heterogeneous reporting. Heterogeneities in nonpolio AFP reporting may also occur at the subnational level, reflecting differential access of populations to healthcare facilities (28), differences in security across the country (26), and local transmission of other infectious causes of AFP, such as nonpolio enteroviruses (29).
The number of reported nonpolio AFP cases has increased over time. We did not account for population growth in the spatiotemporal regression model of nonpolio AFP reporting, but population growth, along with improved surveillance, is likely to be a contributing factor toward the observed increase in reporting in most countries. When testing for AFP clustering in the real-time AFP database, we based the expected incidence of AFP in the absence of an outbreak only upon reported AFP cases in the preceding 2 years from the current week of surveillance. Therefore, the general time trend toward increased reporting of AFP cases would not give lead to false identification of clusters unless there was a large shift in reporting practices during those years. We assume that geographic differences in the incidence of reported AFP cases do not change over time. The low false-alarm rate in the majority of countries suggests that this is a reasonable assumption. However, the spatial random effects in our model could also be updated annually to account for such changes.
During the years of our study, no record was kept of when AFP cases were recorded in a central database. Thus, a limitation of our work is that we assumed that AFP cases were reported to the polio information system on the date the case was notified by local healthcare providers. In practice there may be further delays in collation of local information into the global polio information system AFP database. We compared the date of notification of AFP cases by local healthcare providers with the date of entry into this database by downloading this database every week during 2015 and found a median delay of 25 days (interquartile range 21–32 days) (Figure 11 at http://dx.doi.org/10.5281/zenodo.44361). If this delay were to persist it would postpone the date of an early-warning alarm by the duration of the delay; thus, there is a strong case for faster collation of data into national and global databases.
Polio outbreaks that are detected late will threaten the progress of the GPEI, and consequently there is a need to strengthen ongoing surveillance. Although future work is required to test our algorithm in other settings, we have shown that integrating an automated early-warning system based on detection of AFP clusters into the polio information system could be of value to the GPEI, helping to identify large outbreaks earlier and stop transmission faster.
Dr. Blake is a research fellow in the Department of Infectious Disease Epidemiology, Imperial College London. Her research is focused on using mathematical and statistical methods to understand and quantify the spread of infectious diseases.
We acknowledge the efforts of all persons involved in the epidemiologic investigations and laboratory testing of the polio outbreaks and surveillance. We thank Ananda Bandyopadhyay and Marga Pons-Salort for helpful discussions on this work.
This work was supported by a grant from the Bill and Melinda Gates Foundation and center funding from the Medical Research Council.
- Global Polio Eradication Initiative. Data and monitoring. 2015 [cited 2015 Oct 20]. http://www.polioeradication.org/Dataandmonitoring.aspx
- Aylward RB, Alwan A. Polio in Syria. Lancet. 2014;383:489–91.
- Burns CC, Diop OM, Sutter RW, Kew OM. Vaccine-derived polioviruses. J Infect Dis. 2014;210(Suppl 1):S283–93 and.
- Global Polio Eradication Initiative. Polio eradication and endgame strategic plan 2013–2018. 2013 [cited 2014 Apr 27]. http://www.polioeradication.org/Resourcelibrary/Strategyandwork.aspx
- Centers for Disease Control and Prevention. Progress toward interrupting wild poliovirus circulation in countries with reestablished transmission—Africa, 2009–2010. MMWR Morb Mortal Wkly Rep. 2011;60:306–11 .
- Levitt A, Diop OM, Tangermann RH, Paladin F, Kamgang JB, Burns CC, Surveillance systems to track progress toward global polio eradication—worldwide, 2012–2013. MMWR Morb Mortal Wkly Rep. 2014;63:356–61 .
- Melnick JL, Ledinko N. Development of neutralizing antibodies against the three types of poliomyelitis virus during an epidemic period; the ratio of inapparent infection to clinical poliomyelitis. Am J Hyg. 1953;58:207–22 .
- Penttinen K, Patiala R. The paralytic/infected ratio in a susceptible population during a polio type I epidemic. Ann Med Exp Biol Fenn. 1961;39:195–202 .
- Marx A, Glass JD, Sutter RW. Differential diagnosis of acute flaccid paralysis and its role in poliomyelitis surveillance. Epidemiol Rev. 2000;22:298–316.
- Blake IM, Martin R, Goel A, Khetsuriani N, Everts J, Wolff C, The role of older children and adults in wild poliovirus transmission. Proc Natl Acad Sci U S A. 2014;111:10604–9.
- Linard C, Gilbert M, Snow RW, Noor AM, Tatem AJ. Population distribution, settlement patterns and accessibility across Africa in 2010. PLoS ONE. 2012;7:e31743.
- Gaughan AE, Stevens FR, Linard C, Jia P, Tatem AJ. High resolution population distribution maps for Southeast Asia in 2010 and 2015. PLoS ONE. 2013;8:e55882.
- Hijmans RJ, van Etten J, Mattiuzzi M, Sumner M, Greenberg JA, Lamigueiro OP, raster: geographic data analysis and modeling. R package [cited 2014 Apr 27]. https://cran.r-project.org/web/packages/raster/
- R Development Core Team. R: a language and environment for statistical computing [cited 2014 Apr 27]. http://www.R-project.org/
- Besag J, York J, Mollié A. Bayesian image restoration, with two applications in spatial statistics. Ann Inst Stat Math. 1991;43:1–20.
- Rue H, Martino S, Chopin N. Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations. J R Stat Soc Series B Stat Methodol. 2009;71:319–92.
- Rue H, Martino S, Lindgren F, Simpson D, Riebler A, Krainski ET. INLA: functions which allow to perform full Bayesian analysis of latent Gaussian models using integrated nested Laplace approximation. R package version 0.0–1403203700 [cited 2014 Apr 27]. http://www.r-inla.org/download
- Spiegelhalter DJ, Best NG, Carlin BR, van der Linde A. Bayesian measures of model complexity and fit. J R Stat Soc Series B Stat Methodol. 2002;64:583–639.
- Kulldorff M. Prospective time periodic geographical disease surveillance using a scan statistic. J R Stat Soc Ser A Stat Soc. 2001;164:61–72.
- Kulldorff M; Information Management Services Inc. SatScan v9.3: software for the spatial and space–time scan statistics [cited 2014 Jun 12]. http://www.satscan.org/
- Kulldorff M, Heffernan R, Hartman J, Assunção R, Mostashari F. A space–time permutation scan statistic for disease outbreak detection. PLoS Med. 2005;2:e59.
- Wassilak SG, Oberste MS, Tangermann RH, Diop OM, Jafari HS, Armstrong GL. Progress toward global interruption of wild poliovirus transmission, 2010–2013, and tackling the challenges to complete eradication. J Infect Dis. 2014;210(Suppl 1):S5–15.
- Mostashari F, Kulldorff M, Hartman JJ, Miller JR, Kulasekera V. Dead bird clusters as an early warning system for West Nile virus activity. Emerg Infect Dis. 2003;9:641–6.
- Heffernan R, Mostashari F, Das D, Karpati A, Kulldorff M, Weiss D. Syndromic surveillance in public health practice, New York City. Emerg Infect Dis. 2004;10:858–64.
- Reinhardt M, Elias J, Albert J, Frosch M, Harmsen D, Vogel U. EpiScanGIS: an online geographic surveillance system for meningococcal disease. Int J Health Geogr. 2008;7:33.
- Alleman MM, Meyer SA, Mulumba A, Nyembwe M, Riziki Y, Mbule A, Improved acute flaccid paralysis surveillance performance in the Democratic Republic of the Congo, 2010–2012. J Infect Dis. 2014;210(Suppl 1):S50–61.
- Khuzwayo LS, Kuonza LR, Ngcobo NJ. Evaluating the acute flaccid paralysis surveillance system in South Africa, 2005–2009: an analysis of secondary data. Pan Afr Med J. 2013;14:86 .
- Macama A, Okeibunor J, Grando S, Djibaoui K, Yameogo RK, Morais A, Reasons and circumstances for the late notification of acute flaccid paralysis (AFP) cases in health facilities in Luanda. Pan Afr Med J. 2014;18:239.
- Grist NR, Bell EJ. Paralytic poliomyelitis and nonpolio enteroviruses: studies in Scotland. Rev Infect Dis. 1984;6(Suppl 2):S385–6.