During case investigation interviews in Pennsylvania, digital notification app users were identified and given a validation code to enter into their app. The app then automatically sent anonymous notifications to other users identified through smartphone Bluetooth technology as potentially exposed to the person testing positive for COVID-19 (Appendix).

The Pennsylvania Department of Health (PA DoH) collected data on the performance of standard CICT and digital notification apps (Table). We aggregated those data across all counties, excluding Philadelphia County (Appendix), for 8 weeks, November 8, 2020–January 2, 2021 (Table). We extracted the daily number of COVID-19 cases from the Centers for Disease Control and Prevention (CDC) COVID Data Tracker (8).

We used CDC’s COVIDTracer modeling tool to estimate cases and hospitalizations averted by digital notifications during the 8-week study period (1,2,9). COVIDTracer uses an epidemiologic model to illustrate the spread of COVID-19 and effects of CICT and other nonpharmaceutical interventions (NPIs). We calculated a summary effectiveness measure for CICT and digital notification apps from the various data PA DoH collected and input this measure to the model (Table). We defined this summary effectiveness measure as the proportion of cases that entered isolation and contacts that quarantined in response to CICT and digital notification apps, and the number of days required to do so (i.e., number of days from exposure to isolation or quarantine). We further assumed 60%–100% of interviewed cases and monitored contacts fully adhered to isolation and quarantine guidelines, and that 10%–50% of notified but not monitored contacts complied with quarantine guidance (10–12). To calculate the number of days from exposure to isolation or quarantine, we averaged the number of days between case interviews (triggering case isolation) and contact notifications (triggering contact quarantine). We performed 2 sensitivity analyses by varying the estimated number of days from infection to isolation by +1 day and the weight used to estimate the overall proportion of cases isolated and contacts quarantined (Appendix).

We derived CICT program effectiveness from reported data, but data were not available to estimate effectiveness of other NPIs, such as social distancing and mask-wearing. Therefore, we used the tool to estimate the effectiveness of other NPIs by fitting the model-generated curve to observed case curve (Appendix). Finally, to show what might have happen without the digital notifications, we simulated a hypothetical case curve by replacing the CICT effectiveness input with a value excluding contributions of the digital notifications. We considered the difference between cases in the simulated curve and reported cases as the estimated cases averted by the digital notifications. We generated a range of 18 results by varying public compliance with isolation and quarantine guidance and the degree to which recipients of digital notifications were also notified by the PA DoH staff members. First, we assumed no overlap (i.e., all digital notifications were sent to contacts who were not notified by the DoH staff); then, we assumed a 50% overlap (Appendix Tables 4, 5). We also calculated the number of hospitalizations averted by multiplying the estimated number of averted cases by age-stratified infection-to-hospitalization rates (9). We did not account for vaccination because only 0.1% of Pennsylvania’s population was fully vaccinated during the study period.

Between its launch in late September and the end of the study period, Pennsylvania’s digital notification app was downloaded 638,797 times, accounting for ≈5.7% of the population; 56% (n = 356,835) of downloaded apps were actively used, accounting for 3.2% of the population. In all, 786 interviewed case-patients (0.2% of all cases) had the digital notification app installed on their smartphones, among whom <50% (n = 390) used the app to notify others of potential exposure, totaling 233 digital notifications during the 8-week period (Table).

Figure 1

Figure 1. Estimated number of cases averted per 1,000 COVID-19 digital notifications, Pennsylvania, USA, November 8, 2020–January 2, 2021. Estimates show selected scenarios of isolation or quarantine compliance and the digital notification...

We estimated those digital notifications averted 2–16 additional cases (7–69 cases/1,000 notifications) and <1 hospitalization (Figure 1; Appendix Tables 4, 5). That range reflects uncertainties in both public compliance and the degree of overlap between notifications received via the digital notification app and DoH staff. In comparison, we estimated standard CICT averted 10,168–17,151 cases and 250–421 hospitalizations during the same period.

Although just 3.2% of the state’s population used the COVID Alert PA app, we estimated that 7–69 cases were averted for every 1,000 digital notifications sent during the 8-week study. Those estimates represent a single locality and should not be generalized to other jurisdictions. However, the methods, and the publicly accessible modeling tool, could be used to adjust for differences in uptake, compliance, and epidemic curve to estimate the effect of digital notifications in other jurisdictions.

Figure 2

Figure 2. Overlap between standard CICT and digital notifications in a study of estimated cases averted by COVID-19 digital exposure notification, Pennsylvania, USA, November 8, 2020–January 2, 2021. During the study period,...

Greater use, increased compliance, or changes to digital notification system operations might increase its effectiveness (Figure 2). UK researchers assessing a similar app estimated that 167–349 cases were averted for every 1,000 notifications with a 28% adoption rate (13). Greater use appears achievable based on multiple reports indicating >17% of the population activated digital notification apps in 11 states and participation approached 50% in states where adoption was greatest (6,7). When we examined hypothetical scenarios in which 50% of the population actively used the app in Pennsylvania, all else remaining equal, we found that up to 3,995 cases could have been averted by digital notifications during the study period (Appendix).

The potential increase in cases averted by digital notifications requires additional research and should consider other factors, such as alternative digital notification system operations. For example, effectiveness might be improved with automatic digital notification versus relying on case-patients to initiate contact notification after being interviewed. Some jurisdictions also started permitting users to self-report as COVID-19–positive and initiate digital notifications on the basis of at-home testing, which could improve both the number and timeliness of digital notifications (14). Although such gains are promising, they are moderated by the public’s compliance with digital notifications and technological limitations of Bluetooth signaling, leading to missed exposures and potentially false notifications.

Our findings suggest that the use of digital notification apps helped avert COVID-19 cases in Pennsylvania, although its effectiveness was limited by numerous factors, most notably limited use. The results also suggest opportunities exist to further examine and improve digital notification systems and their use during future outbreaks (Figure 2). Public health practitioners should explore ways to increase public participation in digital notification apps and to improve system efficiency by increasing the timeliness, coverage, and accuracy of digital notifications.