Scientific Understanding

For forecasting using mathematical and statistical models, it is critical to understand the key processes affecting AMR spread. Those processes are often represented as nonlinear effects in forecasting models and, if not properly specified, will produce forecasts that quickly diverge from the truth. As of 2023, many questions on AMR remain open (27). For instance, the role of antibiotic use in driving AMR is not fully understood, particularly the effects of co-selection (i.e., selection of resistance that is broader than the specific target of an antimicrobial prescription) (28) and the relationship between outpatient use of antimicrobial drugs and resistant infections of hospitalized patients. More generally, it is not yet known which type of antimicrobial drug use (e.g., community use, hospital use, or veterinary use) has the greatest effect on AMR emergence (29). After the emergence of AMROs, it is unclear how competition with susceptible strains affects the incidence of resistant strains and how to explain their coexistence over long time periods (30). Likewise, the issue of spillover (i.e., transmission of AMR across locations) is arguably a substantial challenge for forecasting that has not been addressed (31).

In healthcare facilities, it is unknown how contact networks and heterogeneity of exposure to antibiotics shape the spread of AMR; it is hard to disentangle the roles of community importation and nosocomial transmission; and it is difficult to quantify the relative transmissibility among classes of persons (patients, healthcare workers) and the environment. In addition, individual-level causal relationships between the type and duration of therapy and resistance emergence remain unknown in most instances. The human microbiome serves as a reservoir of antimicrobial resistance (32–34); however, many outstanding scientific questions on microbiome effects are still under active research as of January 2023. Further studies are needed to examine the role of bystander selection (i.e., selection of resistance on microbes that are not the target pathogen) in AMR emergence (35,36), the reason treatment with cephalosporins is a risk factor for vancomycin-resistant Enterococcus colonization (37), and the difference between detectable colonization and high-level colonization.

To date, infectious disease forecasting has primarily focused on acute viral infections for which the pathogen and its disease or clinical outcome can be directly linked. For instance, viral load is generally correlated with infectivity and disease phenotype (mild to severe) and, therefore, with illness and death rates. Those correlations make definition of the forecasting target (e.g., incident rates of cases, hospital admissions, or deaths) relatively straightforward. However, for bacterial or fungal species, relationships between pathogen load and clinical outcomes are unclear. Because many bacterial species are commensal with their human host and have varying probabilities of presence across body sites, it is challenging to definitively determine whether a person is colonized. Without accurate observation of colonization, AMR burden is not well resolved and, consequentially, is more difficult to forecast.

Accessing High-Quality Data

Forecasting is fundamentally a data-driven task. Without sufficient data, predictive models cannot be properly trained and evaluated. As of 2023, data that can inform operationally useful forecasts of AMR remain scarce. At the population level, several surveillance systems do exist. For instance, the US National Antimicrobial Resistance Monitoring System for Enteric Bacteria tracks changes in antimicrobial susceptibility for certain enteric bacteria in ill persons, retail meats, and food animals (38). However, consistent long-term records of AMR pathogen profiles are lacking in most countries, particularly in low- and middle-income countries and for emerging AMROs with limited cases (39). In addition, several major pathogens responsible for healthcare-associated infections have not been included in surveillance.

At the facility level, AMR data from EHR have become increasingly available to researchers in recent years. In healthcare settings, more attention has been given to infected patients with clinical manifestations. Surveillance for asymptomatic AMRO carriage is not prioritized because it is not of immediate clinical interest, although such carriers play an important role in onward transmission (20). Such incomplete observation hinders estimation of overall AMRO prevalence and may lead to biased prediction targets. In addition, data on nonbiologic processes driving AMR pathogen transmission, such as patient behavior and interactions with healthcare workers, are difficult to collect. In cases for which relevant data are available, data quality may be poor because records can include errors and misclassification. Even for structured EHR data, both predictive variables and outcomes (e.g., colonization) can suffer from missing data.

Model Calibration

Model calibration is the process by which a mathematical model is tuned to reproduce empirical observations. Although this process does not guarantee accurate prediction, model calibration provides an initial check that the model can closely replicate historical data. Studies that calibrate AMR models to empirical data have been published (23,24,40–42). However, as the structure of AMR models becomes increasingly complex, computational difficulties arise in fitting these models to observations of different types and at various scales. For instance, population-level prevalence, individual-level test results, and genomic sequences of pathogens convey different pieces of information on AMRO transmission, and calibrating AMR models to these observations simultaneously is a challenge. AMRO transmission is intrinsically stochastic with large uncertainty. Quantifying the uncertainty of predictions generated by complex AMR models is difficult, especially for models that track individual persons and their contacts. Calibration approaches, and their success, usually depend on the specific model construct and the form of observations.

Implementation and Evaluation

One prominent challenge for AMRO forecasting is the operational implementation and prospective evaluation of predictive models (i.e., generating forecasts in real time and evaluating those forecasts once prediction targets are observed). There are no guidelines on such implementation for AMRO forecasting, such as appropriate data collection and forecast targets. Questions remain open on the proper time scale of forecast horizon, the frequency at which models need to be updated, and the fundamental limit of predictability of models. For long-lead forecasting, evaluation requires data collection in a consistent manner over a long time period. In healthcare facilities, the practice of testing and reporting AMR infections may change over time, which further complicates using such data records and forecast evaluation. A collaborative effort that standardizes training datasets, forecast targets, forecast horizons, and proper scoring rules for evaluating forecast performance (e.g., the FluSight influenza forecasting challenge [43–45], the dengue forecasting challenge [4], and the RAPIDD Ebola forecasting challenge [46]) can potentially stimulate advances in operational AMR forecasting.

A particular challenge for implementing AMRO forecasting is to handle uncertainty in predictions; uncertainty exists because of imperfect data and a notable degree of variability in many AMR-related processes. Quantifying such uncertainty is critical in other predictive fields, such as numerical weather prediction. For AMROs, whether at the facility level (e.g., determining which patients need to be on contact precautions) or the community level (e.g., public health officials making recommendations for prescribing guidelines because of AMR), decision makers must make decisions that leverage uncertain information. This truism holds for observations as well as forecasts. Designing optimal decision frameworks and architectures that best use forecasts, given their uncertainty, is a needed long-term goal.

Effective communications between modelers and stakeholders such as public health officials, healthcare institutions, and individual practitioners are critical to learn their practical needs from AMR modeling. However, formal reports recording such communications are lacking in scientific journals, which is another factor limiting the generation and use of operational forecasts in real-world settings.

Figure

Figure. Open questions for predictive modeling of MRSA. Example questions at the facility level, the population level, and across scales are listed. The upper left panel depicts population-level and facility-level MRSA...

To illustrate the interconnected challenges faced by AMRO forecasting across scales, we use methicillin-resistant Staphylococcus aureus (MRSA) as a concrete example (Figure). Several key issues on MRSA forecasting at the facility scale and population scale and across scales are unresolved; one is that the specific data needed for modeling at those different scales are unknown, as is the role of co-selection and competition with methicillin-susceptible S. aureus (MSSA) in affecting the dynamics of MRSA. Answering those questions would improve methods to reduce MRSA burden in both community and hospital settings.

In this perspective, we focus on real-time forecasting of AMROs. A parallel line of research is scenario-based simulations that project AMR infections conditional on postulated changes in prescribed interventions or expected conditions. Previous studies for HIV and tuberculosis control show that such scenario-based projections can substantially affect health policies and save lives (47,48). For AMR, scenario-based projections should be designed to address practical questions in public health and inform operational policy decision-making in real time, possibly using ensemble approaches that combine multiple models to reflect cross-model variation. Real-time forecasting and scenario-based projections complement each other and should be developed in tandem to control AMR burden and improve human health.