In Defense of Defensive Forecasting

2025-06-13

AI Summary

This paper addresses the limitations of traditional forecasting methods—namely, their reliance on probabilistic modeling assumptions and hypothetical future distributions—by proposing a hypothesis-free, non-asymptotically optimal defensive prediction framework. Instead of assuming a data-generating mechanism, the method formulates forecasting as an adversarial sequential game and achieves robustness via online correction of past prediction errors. Its key contributions are threefold: (1) a unified defensive prediction paradigm grounded in Vovk’s game-theoretic framework and online convex optimization; (2) a hyperparameter-free recursive calibration algorithm, providing the first theoretical guarantees for online conformal prediction; and (3) attainment of the optimal $O(sqrt{T})$ regret bound, strong calibration, and exact calibration under arbitrary sequences. Experiments demonstrate its simplicity and near-optimal performance across online learning, expert aggregation, and conformal prediction tasks.

Technology Category

• Game Theory and Economic Paradigms: Adversarial Learning
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Adversarial Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

This tutorial provides a survey of algorithms for Defensive Forecasting, where predictions are derived not by prognostication but by correcting past mistakes. Pioneered by Vovk, Defensive Forecasting frames the goal of prediction as a sequential game, and derives predictions to minimize metrics no matter what outcomes occur. We present an elementary introduction to this general theory and derive simple, near-optimal algorithms for online learning, calibration, prediction with expert advice, and online conformal prediction.

Problem

• Algorithms for Defensive Forecasting derived by correcting past mistakes
• Predictions minimize metrics regardless of outcomes in sequential game
• Simple near-optimal algorithms for online learning and prediction tasks

Innovation

• Corrects past mistakes for predictions
• Minimizes metrics in sequential game
• Provides near-optimal online learning algorithms