Volume 26, Number 11—November 2020
Research
Nowcasting (Short-Term Forecasting) of Influenza Epidemics in Local Settings, Sweden, 2008–2019
, Olle Eriksson, Örjan Dahlström, Benjamin John Cowling, Matthew Biggerstaff, Gunnar Ljunggren, Anna Jöud, Emanuel Istefan, and Toomas Timpka
Abstract
The timing of influenza case incidence during epidemics can differ between regions within nations and states. We conducted a prospective 10-year evaluation (January 2008–February 2019) of a local influenza nowcasting (short-term forecasting) method in 3 urban counties in Sweden with independent public health administrations by using routine health information system data. Detection-of-epidemic-start (detection), peak timing, and peak intensity were nowcasted. Detection displayed satisfactory performance in 2 of the 3 counties for all nonpandemic influenza seasons and in 6 of 9 seasons for the third county. Peak-timing prediction showed satisfactory performance from the influenza season 2011–12 onward. Peak-intensity prediction also was satisfactory for influenza seasons in 2 of the counties but poor in 1 county. Local influenza nowcasting was satisfactory for seasonal influenza in 2 of 3 counties. The less satisfactory performance in 1 of the study counties might be attributable to population mixing with a neighboring metropolitan area.
Reliable forecasts of the timing and spatial spread of influenza during seasons and pandemics can meaningfully advance the timing of public health communication campaigns and implementation of resource allocation in healthcare (1). Different types of influenza forecast methods have been developed and applied to support public health response (2). However, although modelers have shown considerable interest in developing infectious disease forecasts, the readiness in the public health community for applying these predictions has been lacking (3). One reason for this discrepancy might be that national public health policies for response to infectious disease outbreaks often assign the responsibility for healthcare resource allocation to local health authorities (i.e., county and municipality governments). For geographic and infrastructural reasons, the timing of the spatial spread of influenza can differ substantially between these administrative units within nations and states. Therefore, a need exists for influenza forecasting methods that harmonize with policy-making responsibilities at local government levels and that are more relevant for public health practitioners.
Another reason for the poor uptake of forecasting methods might be a lack of prospective evaluations of their reliability. To address this issue, the US Centers for Disease Control and Prevention (CDC) has run the Forecast the Influenza Season Collaborative Challenge (FluSight) since the 2013–14 influenza season to prospectively evaluate different methods and data sources for influenza forecasting at the national, regional, and (starting in the 2017–18 influenza season) state level (4). At the local (county and municipality) level, however, few corresponding prospective evaluations based on routine health system data have been reported. Short-term forecasting is denoted as nowcasting (5). Recently, a prospective 5-year appraisal of a local nowcasting method (6) in a county in Sweden (county population ≈460,000) indicated promising results with regard to detection of the local start of the epidemic, prediction of peak timing, and prediction of peak intensity (7). The appraisal concluded that a longer prospective evaluation was needed to ascertain the validity of the results and that data from larger urban counties were required to draw reliable conclusions about generalizability.
In this article, we describe a prospective 10-year evaluation of this local influenza nowcasting method in 3 urban counties (population 1.3–2.2 million) in Sweden. The evaluation period included 1 pandemic (2009) and 9 seasonal influenza epidemics.
Study Design
Figure 1
Figure 1. Three regions analyzed in study of nowcasting for influenza epidemics in local settings, Sweden. Black indicates Stockholm County, red West Gothia County, gray Scania County. Included in the map is...
Figure 2
Figure 2. Daily numbers of influenza-diagnosis cases per 100,000 population, January 1, 2008–February 28, 2019, in Stockholm County (upper graph), West Gothia County (middle graph), and Scania County (lower graph), Sweden.
Figure 3
Figure 3. Daily numbers of telenursing calls attributable to fever (among children and adults) per 100,000 population, January 1, 2008–February 28, 2019, in West Gothia County (upper graph) and Scania County (lower...
We used an open cohort design based on the total population in 3 urban counties: Stockholm County (population 2,231,000), West Gothia County (population 1,649,000), and Scania County (population 1,304,000) (Figure 1). We used retrospective data from January 1, 2008, through June 30, 2009, and prospective data from July 1 through February 28, 2019, from 2 sources in the countywide health information system: daily numbers of clinically diagnosed influenza cases (Figure 2) and daily syndromic chief complaint data from a telenursing service (Figure 3) (6,7). The clinical influenza case data were used for detection of the local start of the epidemic and prediction of its peak intensity, and the syndromic data were used to detect the peak timing. Existing evidence of to predict strong association between the clinical influenza case data and syndromic chief complaint data from the telenursing service was used in this nowcasting method (8,9). Because of a change of system, no syndromic chief complaint data were available for Stockholm County. Syndromic data from West Gothia County were therefore used to predict of the peak timing for Stockholm County.
Timeliness was used as a performance metric for detection of the local start of the epidemic and the peak-timing prediction; the correct identification of intensity category on a 5-grade scale (i.e., nonepidemic, low, medium, high, and very high) was used for peak-intensity prediction. The study design was approved by the Regional Research Ethics Board in Linköping (approval no. 2012/104-31).
Definitions
Influenza cases were identified by using codes from the International Classification of Diseases, 10th Revision, for influenza (J10.0, J10.1, J10.8, J11.0, J11.1, J11.8) (10) as recorded in the local electronic health data repository. Only influenza diagnoses in the first coding position were used. Influenza-related telenursing call cases were identified by using the chief complaint codes associated with influenza symptoms. The symptoms used were fever, cough, and headache. These data were downloaded from the electronic patient record systems to the electronic health data repository twice daily.
The intensity level defining the start of a local epidemic (i.e., the intensity that determines the endpoint for the detection function) was set to 6.3 influenza-diagnosis cases/100,000 population recorded during a floating 7-day period in the countywide health information system (6). This level was chosen by inspecting the epidemic curves of previous local influenza epidemics. A recent comparison of influenza intensity levels in Europe estimated a similar level (6.4 influenza-diagnosis cases/week/100,000 population) for the 2008–09 seasonal influenza in Sweden (11). The optimal alerting threshold before each epidemic was decided by calculating the sensitivity and the specificity for the previous nonpandemic influenza seasons and studying them on a receiver operating characteristic curve (6). The calculation of the specificity was based on all days in the nonepidemic period (i.e., before the limit of 6.3 influenza-diagnosis cases/100,000 during a floating 7-day period occur), and the calculation of the sensitivity was based on the days in the epidemic period (i.e., from when the limit of 6.3 influenza-diagnosis cases/100,000 during a floating 7-day period has occurred). Peak timing was defined as the day when the highest number of influenza-diagnosis cases were documented in the countywide electronic patient record. Peak intensity was defined as the number of influenza-diagnosis cases that had been documented at peak timing.
Method Application
Technical details concerning the 3 functions of nowcasting have been described previously (6; Appendix). The functions are detecting the start of the influenza season or pandemic and forecasting the peak day and peak intensity. Once the epidemic has been detected using the clinical influenza data, the syndromic telenursing data are used to detect when it decreases, that being the indication for the peak. Because changes in clinical influenza data have been found to occur 14 days after corresponding changes in syndromic data, the peak timing in the clinical influenza data are forecasted to occur 14 days after the peak in the syndromic data. Finally, the peak intensity is forecasted by using the clinical influenza data. Syndromic data have a higher amplitude, and the relationship between syndromic data and clinical influenza data are not necessarily constant between seasons. Therefore, the clinical data were used to predict the intensity once the peak day is predicted with the help of syndromic data.
To calibrate the detection component of the nowcasting method, we retrospectively determined weekday effects on recording of influenza-diagnosis cases and a baseline alert threshold by using the retrospective data. These data were collected from January 1, 2008, through June 30, 2009, including 2 nonpandemic influenza seasons (2007–08 and 2008–09). To determine weekday effects, data from the entire retrospective data collection period were used. To determine the initial alert threshold, only data from the seasonal influenza in 2008–09 were used. The 2007–08 seasonal influenza could not be used for this purpose because the season had started before January 1, 2008. Throughout the study period, the calibration data were updated after every seasonal influenza (i.e., no updates of the threshold after the 2009 pandemic outbreak). The detection algorithm was thus applied to the next epidemic by using the revised threshold determined in the updated retrospective dataset.
Before the 2010–11 seasonal influenza, no updates were performed because the set of retrospective data remained the same (i.e., it contained data from the 2008–09 seasonal influenza but excluded pandemic data). For the 2011–12 seasonal influenza, the threshold was updated by using retrospective data from the 2008–09 and 2010–11 seasonal influenza. For the 2012–13 seasonal influenza, the threshold was updated by using retrospective data from the 2008–09, 2010–11, and 2011–12 seasonal influenza, and so on. The weekday effects were assumed to be relatively constant over time in the local detection analyses and were therefore not updated after every seasonal influenza.
The set of retrospective data from the seasonal influenza in 2007–08 and in 2008–09 were also used to initially calibrate peak-timing prediction for West Gothia County and Scania County. The dataset was used to decide the grouping of chief complaints with the largest correlation strength and longest lead time from telenursing data to influenza-diagnosis data (10,11). For both counties, the best performing telenursing chief complaint was fever (among children and adults), and the most favorable lead time was 14 days. When the peak timing had been determined, the second component of the local prediction module was applied to influenza-diagnosis data from the corresponding epidemics to find the peak intensity on the predicted peak day (6). Regarding weekday effects on local prediction, the same calculation was applied and the same grouping of chief complaints and lead time were used throughout the study.
Metrics and Interpretations
On the basis of the utility of the nowcasting method in local healthcare settings, the maximum tolerable timeliness error for detection and peak-timing predictions was set to 11 days (»1.5 weeks). Method performance was defined to be excellent if the absolute value of the timeliness error was <3 days, good if it was 4–7 days, tolerable if it was 8–11 days, and poor if it was >12 days. For the interpretation of peak intensity predictions, the intensity level categories (nonepidemic <0.9, low 0.9, medium 2.4, high 5.5, and very high intensity level 7.9 cases/day/100,000 population) identified using the moving epidemic method for the reference influenza season 2008–09 in Sweden (11) were used. If the predicted peak intensity fell into the same category as the recorded peak intensity, the prediction was considered excellent. If the predicted peak intensity did not fall into the same intensity category, the predicted peak was considered good if it was up to 10% above or below the threshold for the recorded peak intensity category, tolerable if the predicted peak was 10%–20% above or below the threshold for the recorded peak intensity category, and poor otherwise. When assessing series of nowcasts, the performance of a sequence of nowcasts was considered satisfactory if all separate forecasts were assessed as excellent, good, or tolerable, and poor otherwise.
Local Detection
The date of the actual start of the epidemic phase for the 10 influenza epidemics differed by 2–27 days between the 3 counties (Table 1). The detection component of the local nowcasting method showed good or excellent performance in all counties under surveillance for 6 of the 9 nonpandemic influenza seasons and in 2 out of 3 counties under surveillance for the 3 remaining seasons. Twice the poor alerts were issued too soon and once belatedly. The detection performance was good during the 2009 influenza A(H1N1) pandemic in 2 of 3 counties (Stockholm and West Gothia) and poor in 1 county (Scania).
Local Prediction
For the 2009 influenza pandemic, the performance of the peak-timing prediction was poor in all 3 study counties (Table 2). The peak-timing prediction was also poor for the 2010–11 seasonal, when influenza A(H1N1) and B viruses were circulating. Thereafter, the predictions were tolerable for the 2011–12 seasonal influenza, when influenza A(H3N2) virus was circulating, and good to excellent for the remaining influenza seasons, with the exception of the poor peak-timing predictions for Scania County for the 2016–17 and 2017–18 influenza seasons, with influenza A(H3N2) virus circulating in 2016–17 and influenza A(H3N2) and B in 2017–18.
The prediction of the peak-intensity level was poor for the 2009 influenza pandemic in all 3 study counties (Table 2). For seasonal influenza, in 2 of the study counties (Stockholm and West Gothia) the predictions were tolerable to excellent for all seasons, except for the 2018–19 season with influenza A(H1N1) in Stockholm. In 1 county (Scania), the peak-intensity predictions were poor for 5 of the 9 influenza seasons: 2010–11 with influenza A(H1N1) and B, 2011–12 with influenza A(H3N2), 2014–15 with influenza A(H3N2) and B, 2015–16 with influenza A(H1N1) and B, and 2017–18 with influenza A(H3N2) and B circulating.
Epidemic forecasts for large administrative areas (e.g., nations or states) might not be sufficiently informative for local response to epidemics if sizable variations in disease transmission patterns exist between the smaller administrative areas (e.g., counties) with independent local healthcare governance that they contain (12). The importance of taking the local context into regard in epidemic forecasting has been further emphasized during the current coronavirus pandemic (13). In our prospective 10-year evaluation of local nowcasting in 3 urban counties, the start of the influenza seasons included differed by up to 27 days and the peak intensity by >1 intensity level among the counties, whereas the time-of-peak differences were small. The purpose of the evaluated local detection function was to allow hospitals and primary healthcare centers time to prepare for management of influenza patients (e.g., by preparing intensive care unit resources or postponing some elective procedures). This component showed satisfactory performance in all 3 counties. The peak-timing prediction function was aimed at informing the local authorities when the peak has occurred and that health service routines soon can be permitted to return to normal. This component showed satisfactory performance from the 2011–12 influenza season onward. Predictions of peak timing were made 8–10 days before the peak and were +7 days accurate in most cases. This finding contrasts with the current practices in the study counties, where the peak of an influenza season is retrospectively determined from surveillance data »10–14 days after it has occurred. The nowcasting of peak-intensity level was aimed at warning the local authorities about high-intensity influenza transmission and the potential need for social distancing measures (e.g., closure of kindergartens). This component provided satisfactory information for influenza seasons in 2 out of 3 study counties (Stockholm and West Gothia).
Although the evaluated nowcasting method is automated to run on routinely collected healthcare data, the accuracy of the nowcasts depends on the stability of the data supply and information infrastructure over time. The method does not require influenza cases to be confirmed by a laboratory as long as data recording remains relatively stable. Nonetheless, some observations can be made about the sensitivity of the local nowcasts to contextual factors. In Sweden, vaccination adapted to the current circulating strains is made available free-of-cost to the elderly and risk groups before every influenza season. However, in the case of the 2009 influenza A(H1N1) pandemic, a national vaccination campaign was implemented, covering the entire population. This intervention probably influenced the nowcasting performance during the corresponding period. Looking only at the performance for seasonal influenza, we observed outcomes in 1 of the 3 study counties (Scania) the raise concerns about vulnerability of the nowcasts to sociodemographic dynamics (14). Malmö (population 450,000; capital of Scania County, Sweden) and Copenhagen (population 2 million; capital of Denmark) are connected by a bridge providing for daily commuting between the metropolitan areas, and their labor markets are closely integrated. The epidemic situation in the highly cosmopolitan Copenhagen region might have had a stronger influence on influenza epidemics in Scania County than the epidemic situation in the neighboring regions had on the other study counties. By structured introduction, evaluation, and modification of prediction models that use additional data sources and statistical methods, local nowcasting can be adapted also to communities with unusual characteristics (15,16). This evidence-based strategy means that, our method can be incrementally adapted to modeling of, for instance, local rural or semirural communities in which residents commute extensively to a neighboring city that is not included in the model.
Some possible limitations exist in terms of the design of this prospective evaluation that require attention. First and foremost, whether the framework used to interpret the nowcasting performance is adequate from the local health authority perspective should be assessed. Regarding the time-of-peak predictions, the ongoing FluSight study uses weekly data (4), thus accepting forecasts made at a weekly resolution. The evaluation framework used to classify forecasts as excellent was at a higher temporal resolution (less than one half week). This boundary was defined from a county government perspective, where the attention is on local resource allocation (e.g., intensive-care unit facilities and hospital beds) for the care of influenza patients. In this situation, nowcasts that are off by days to weeks might have severe consequences for patients in need of these resources. Categories that are suitable for evaluation of usefulness in local response preparations might not be suitable for interpretation of utility in national or international response planning. These observations suggest that the requirements on the accuracy of peak-timing predictions are context-dependent and warrant further research. Concerning the predictions of peak intensity, evaluation of the peak-intensity forecasts indicated that 22% (6 of 27) of the seasonal influenza nowcasts were poor. Retrospectively documenting baseline and threshold values for influenza epidemics helps define whether an influenza epidemic has been different in intensity compared with previous seasons and thereby contributes to future preparedness planning (17,18). For the evaluation of intensity predictions in this study, we used the thresholds established using the moving epidemic method from the reference 2008–09 seasonal influenza season. To improve the validity of the assessments, annual updates of the threshold values using county-level data from previous seasons should be considered for future evaluations of local influenza nowcasting.
Longitudinal prospective evaluations might be needed to draw valid conclusions concerning the performance of local epidemic nowcasting, and inclusion of data from urban counties might be required for generalizability (7). We found in our study that the performance of seasonal influenza nowcasting was satisfactory during a 10-year period in 3 urban counties regarding local detection and peak-timing prediction performance. The predictions of the local peak-intensity level were satisfactory in 2 of the study counties but poorer in 1 county, possibly because of sudden sociodemographic changes. We conclude that the performance of the local nowcasting method was satisfactory for seasonal influenza. The results are of general interest for local healthcare planning during epidemics because the precision by which healthcare systems can adapt its resources to the management of infected patients in these situations affects the resource availability for all other patient groups.
Dr. Spreco is a researcher in the field of syndromic surveillance at Linköping University and Region Östergötland, Sweden. His main research focus is on evaluation and development of algorithms for detection and prediction of infectious diseases.
Acknowledgments
This study was supported by grants from the Swedish Civil Contingencies Agency (grant no. 2010-2788) and the Swedish Research Council (grant no. 2008-5252). The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Authors’ contributions: A.S., O.E., Ö.D., B.J.C., M.B., and T.T. conceived and designed the study; A.S., O.E., and Ö.D. analyzed the data; A.S., O.E., Ö.D., G.L., and T.T. contributed materials and analysis tools; A.S. and T.T. wrote the paper; Ö.D., O.E., B.J.C., M.B., G.L., A.J., and E.I. revised the manuscript and provided intellectual content; and A.S., O.E., Ö.D., B.J.C., M.B., G.L., A.J., E.I., and T.T. gave final approval of the version to be published. T.T. is guarantor of the content.
References
- Nsoesie EO, Brownstein JS, Ramakrishnan N, Marathe MV. A systematic review of studies on forecasting the dynamics of influenza outbreaks. Influenza Other Respir Viruses. 2014;8:309–16. DOIPubMedGoogle Scholar
- Wu JT, Ho A, Ma ES, Lee CK, Chu DK, Ho PL, et al. Estimating infection attack rates and severity in real time during an influenza pandemic: analysis of serial cross-sectional serologic surveillance data. PLoS Med. 2011;8:
e1001103 . DOIPubMedGoogle Scholar - Viboud C, Vespignani A. The future of influenza forecasts. Proc Natl Acad Sci U S A. 2019;116:2802–4. DOIPubMedGoogle Scholar
- Reich NG, Brooks LC, Fox SJ, Kandula S, McGowan CJ, Moore E, et al. A collaborative multiyear, multimodel assessment of seasonal influenza forecasting in the United States. Proc Natl Acad Sci U S A. 2019;116:3146–54. DOIPubMedGoogle Scholar
- Schmid F, Wang Y, Harou A. Nowcasting guidelines—a summary. Geneva: World Meteorological Organization; 2019 [cited 2019 Jul 15]. https://public.wmo.int/en/resources/bulletin/nowcasting-guidelines-%E2%80%93-summary
- Spreco A, Eriksson O, Dahlström Ö, Cowling BJ, Timpka T. Integrated detection and prediction of influenza activity for real-time surveillance: algorithm design. J Med Internet Res. 2017;19:
e211 . DOIPubMedGoogle Scholar - Spreco A, Eriksson O, Dahlström Ö, Cowling BJ, Timpka T. Evaluation of nowcasting for detecting and predicting local influenza epidemics, Sweden, 2009-2014. Emerg Infect Dis. 2018;24:1868–73. DOIPubMedGoogle Scholar
- Timpka T, Spreco A, Dahlström Ö, Eriksson O, Gursky E, Ekberg J, et al. Performance of eHealth data sources in local influenza surveillance: a 5-year open cohort study. J Med Internet Res. 2014;16:
e116 . DOIPubMedGoogle Scholar - Timpka T, Spreco A, Eriksson O, Dahlström Ö, Gursky EA, Strömgren M, et al. Predictive performance of telenursing complaints in influenza surveillance: a prospective cohort study in Sweden. Euro Surveill. 2014;19:20966. DOIPubMedGoogle Scholar
- World Health Organization. International statistical classification of diseases and related health problems. 10th revision. Volume 2. Geneva: The Organization; 2010 [cited 2019 Jun 1]. https://www.who.int/classifications/icd/ICD10Volume2_en_2010.pdf
- Vega T, Lozano JE, Meerhoff T, Snacken R, Beauté J, Jorgensen P, et al. Influenza surveillance in Europe: comparing intensity levels calculated using the moving epidemic method. Influenza Other Respir Viruses. 2015;9:234–46. DOIPubMedGoogle Scholar
- Chen Y, Ong JHY, Rajarethinam J, Yap G, Ng LC, Cook AR. Neighbourhood level real-time forecasting of dengue cases in tropical urban Singapore. BMC Med. 2018;16:129. DOIPubMedGoogle Scholar
- García-Basteiro AL, Chaccour C, Guinovart C, Llupià A, Brew J, Trilla A, et al. Monitoring the COVID-19 epidemic in the context of widespread local transmission. Lancet Respir Med. 2020;8:440–2. DOIPubMedGoogle Scholar
- Timpka T, Eriksson H, Gursky EA, Nyce JM, Morin M, Jenvald J, et al. Population-based simulations of influenza pandemics: validity and significance for public health policy. Bull World Health Organ. 2009;87:305–11. DOIPubMedGoogle Scholar
- Soliman M, Lyubchich V, Gel YR. Complementing the power of deep learning with statistical model fusion: Probabilistic forecasting of influenza in Dallas County, Texas, USA. Epidemics. 2019;28:
100345 . DOIPubMedGoogle Scholar - Collins GS, Moons KGM. Reporting of artificial intelligence prediction models. Lancet. 2019;393:1577–9. DOIPubMedGoogle Scholar
- Ly S, Arashiro T, Ieng V, Tsuyuoka R, Parry A, Horwood P, et al. Establishing seasonal and alert influenza thresholds in Cambodia using the WHO method: implications for effective utilization of influenza surveillance in the tropics and subtropics. Western Pac Surveill Response J. 2017;8:22–32. DOIPubMedGoogle Scholar
- Rakocevic B, Grgurevic A, Trajkovic G, Mugosa B, Sipetic Grujicic S, Medenica S, et al. Influenza surveillance: determining the epidemic threshold for influenza by using the Moving Epidemic Method (MEM), Montenegro, 2010/11 to 2017/18 influenza seasons. Euro Surveill. 2019;24:
1800042 . DOIPubMedGoogle Scholar
Figures
Tables
Cite This ArticleOriginal Publication Date: October 14, 2020
Table of Contents – Volume 26, Number 11—November 2020
| EID Search Options |
|---|
Advanced Article Search – Search articles by author and/or keyword.
|
Articles by Country Search – Search articles by the topic country.
|
Article Type Search – Search articles by article type and issue.
|
Please use the form below to submit correspondence to the authors or contact them at the following address:
Armin Spreco, Division for Public Health and Statistics, Region Östergötland, Linköping, Sweden SE-581 91 Linköping, Sweden
Top