Not All Accuracy Is Equal: Prioritizing Diversity in Infectious Disease Forecasting

2025-09-25

AI Summary

In infectious disease forecasting, ensemble methods often yield limited or even subpar performance compared to individual models due to neglect of model diversity. This paper proposes a “diversity-first” paradigm that shifts focus from optimizing individual model accuracy alone to explicitly leveraging complementary modeling for enhanced robustness. Methodologically, we introduce a multi-model ensemble framework grounded in error correlation analysis and dynamic weighted aggregation, systematically optimizing both model selection and fusion strategies. Evaluated on COVID-19 and influenza forecasting tasks, our approach consistently outperforms state-of-the-art ensemble baselines across accuracy, stability, and trend discrimination capability. Results demonstrate that diversity-driven ensemble design not only improves predictive reliability but also offers practical advantages in real-world epidemiological forecasting scenarios.

Technology Category

• Machine Learning: Ensemble Methods
• Search and Optimization: Metareasoning and Metaheuristics
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Ensemble forecasts have become a cornerstone of large-scale disease response, underpinning decision making at agencies such as the US Centers for Disease Control and Prevention (CDC). Their growing use reflects the goal of combining multiple models to improve accuracy and stability versus using a single model. However, recent experience shows these benefits are not guaranteed. During the COVID-19 pandemic, the CDC's multi-model forecasting ensemble outperformed the best single model by only 1%, and CDC flu forecasting ensembles have often ranked below multiple individual models. This raises a key question: why are ensembles underperforming? We posit that a central reason is that both model developers and ensemble builders typically focus on stand-alone accuracy. Models are fit to minimize their own forecasting error, and ensembles are often weighted according to those same scores. However, most epidemic forecasts are built from a small set of approaches and trained on the same surveillance data, leading to highly correlated errors. This redundancy limits the benefit of ensembling and may explain why large ensembles sometimes deliver only marginal gains. To realize the potential of ensembles, both modelers and ensemblers should prioritize models that contribute complementary information rather than replicating existing approaches. Ensembles built with this principle in mind move beyond size for its own sake toward true diversity, producing forecasts that are more robust and more valuable for epidemic preparedness and response.

Problem

• Ensemble disease forecasts underperform despite combining multiple models
• Highly correlated errors limit benefits due to similar training approaches
• Prioritizing complementary information over accuracy improves ensemble robustness

Innovation

• Prioritize diverse models over individual accuracy scores
• Select models providing complementary information to ensemble
• Build ensembles focusing on true diversity not size