We considered a signal outbreak to be identical to a reported outbreak when 1) >1 signal was triggered within the same period as the first and last case belonging to the particular reported outbreak, 2) the signal outbreak was associated with the identical geographic location on the municipal level (1 of the 430 municipalities) as the reported outbreak, and 3) the signal outbreak was associated with the identical pathogen (either Campylobacter spp. or norovirus). Using the data available as of June 1, 2007, we considered the number of reported outbreaks (a minimum of 4 cases because the algorithm cannot detect outbreaks with <4 cases), from week 5 of 2005 through week 4 of 2007.

Figure 1

Figure 1. Example of 1 reported outbreak being detected by 3 signals. In this example, 3 signal outbreaks (S1, S2, S3) can be associated with 1 reported outbreak in same municipality and during...

Figure 2

Figure 2. Example of 1 signal outbreak corresponding to multiple reported outbreaks. In this example, 1 signal outbreak (S1) can be associated with 3 reported outbreaks occuring in same municipality; threshold is reached...

During the study period, 118 and 4,309 outbreaks with >4 cases, associated with the pathogens Campylobacter spp. and norovirus, respectively, had been reported. The AODA had signaled 52 (44.1%) of the 118 reported Campylobacter spp. outbreaks and 2,538 (58.9%) of the 4,309 reported norovirus outbreaks (Table). The probability that a signal outbreak reflected a reported outbreak (positive predictive value of AODA) was lower for Campylobacter spp. than for norovirus: 50 (6.4%) of 781 Campylobacter spp. signal outbreaks and 2,115 (75.5%) of 2,801 norovirus signal outbreaks were associated with reported outbreaks. The AODA may have triggered multiple signals during the outbreak if the threshold level was reached during several consecutive weeks (Figure 1). Of the Campylobacter spp. outbreaks, 3 (6.0%) were each identified by 2 different signals; of the norovirus outbreaks, 727 (28.6%) were identified by multiple signals (2–20 signals per reported outbreak) (Table). Furthermore, 1 signal outbreak could correspond with different reported outbreaks when these occurred in the same local area and during the same period (Figure 2). For Campylobacter spp., 4 (8.0%) of the outbreak signals could correspond to >1 reported outbreak; for norovirus, 760 (35.9%) of the signal outbreaks could correspond to 2–26 reported outbreaks (Table).

Germany´s electronic reporting system for infectious disease outbreaks provided a unique opportunity to compare the triggering of signals through AODA with the reporting of outbreaks identified by local health departments. The probability of an outbreak signal being associated with a reported outbreak was much lower for Campylobacter spp. (6.4%) than for norovirus (75.5%). Furthermore, the fraction of cases as part of a reported outbreak was much lower for Campylobacter spp. (3.3%) than for norovirus (71.4%). Differences in route of transmission likely explain why Campylobacter spp. cases are generally more likely to occur sporadically and why norovirus cases are more likely to be part of an outbreak (6–9). These differences might result in a lower frequency of Campylobacter spp. outbreaks. The AODA might generate a signal when a higher than expected number of single cases is observed in a specific period and location, but this signal is likely to reflect an increased number of sporadic cases; an increased number of norovirus cases is more likely to reflect an occurring norovirus outbreak. An alternative possibility is that local health departments are more inclined to identify, investigate, and report norovirus outbreaks than Campylobacter spp. outbreaks (10). These differences demonstrate the importance of designing AODA specifically for the pathogens under surveillance.

For our analyses we used reported outbreaks as the reference standard by which to evaluate the AODA. Although this outbreak reporting is probably incomplete, we believe that it more closely identifies the true number of outbreaks than does retrospectively identifying outbreaks (11) or simulating outbreaks (3,4). Thus, we believe it generates a better reference standard than that used in previous studies.

Our findings question the usefulness of the AODA because a large number of generated signals were not confirmed by the electronic outbreak reporting from local health departments. Our results suggest that AODAs are not useful for detecting outbreaks on a local level because the outbreaks are detected earlier and investigated by the local health department. AODAs might be more useful for detecting multicounty or even multistate outbreaks, which are more difficult to detect by a single local health department. The latter has been well demonstrated by AODA detection of various foodborne outbreaks in Germany (12,13). National surveillance should focus on the follow-up of signals that indicate potential multicounty or multistate outbreaks. We used the county level for the algorithm because we obtain the reported outbreaks on this level first and we wanted to compare both systems. Our standard algorithms run also on federal and state levels, but that was not the subject of this investigation. To enable local health departments to earlier discover multicounty outbreaks, a new version of SurvNet is being developed. This version will give local health departments the opportunity to include more information on the evidence and also the possibility of linking outbreaks from different counties (2). The Robert Koch Institute, along with the state health departments, will develop a standard operating procedure for how to communicate and follow up on signals generated by the AODA.

Our study suggests that the usefulness of AODA to detect local outbreaks is limited because local health departments generally detect local outbreaks earlier and in more detail than these algorithms. Investment in the development of user-friendly outbreak reporting tools for local health departments might therefore provide better information on outbreaks than extensive refinements of AODAs.