Model-Agnostic Online Certificate-Driven Calibration for Time Series Forecasting Under Distribution Shift

2026-09-25

AI Summary

This study addresses the degradation of prediction reliability in time series forecasting under distribution shift, where temporal dependencies exacerbate performance deterioration. To overcome the limitations of the independent and identically distributed assumption, this work proposes a model-agnostic online calibration framework that establishes a Bayesian domain adaptation theory tailored for non-stationary drifts. Specifically, it leverages martingale concentration inequalities to derive finite-sample certificates employed as regularization terms, and introduces a gated residual head to enable dynamic correction. Extensive evaluations demonstrate that the proposed approach significantly enhances both predictive stability and accuracy across diverse architectures—including convolutional networks, attention-based models, and large language models—under challenging covariate and concept drift scenarios.

Abstract

Time series out-of-distribution generalization requires forecasters to remain reliable when deployment dynamics differ from training conditions due to covariate shift, concept shift, and temporal dependence. Probably Approximately Correct Bayesian domain adaptation provides computable certificates by decomposing target risk into a source risk term, a source-to-target mismatch term, and a complexity term, but standard analyses rely on independent sampling and distributional stability, assumptions that are violated in time series by serial dependence and nonstationary shift. We propose a model-agnostic online martingale Probably Approximately Correct Bayesian framework that yields finite-sample certificates under temporal dependence and distribution shift. The certificate replaces independent-sample concentration with martingale concentration that adapts to loss scale and predictable variation. We use the certificate as a surrogate regularizer for online calibration by training a gated residual Bayesian head on top of a fixed forecasting backbone, producing a corrective update that reverts to the backbone prediction when the gate is closed. Online calibration combines a source risk anchor, a posterior-shift penalty, and a time-adaptive mismatch term computed from target windows observed before forecasting. It follows a predict-then-update protocol in which outcomes become available only after forecasting and are used to update subsequent predictions. Experiments across convolutional, attention-based, and large language model-based forecasters show improved stability and accuracy under covariate and concept shift.

Problem

• time series forecasting
• distribution shift
• out-of-distribution generalization
• PAC-Bayesian
• online calibration

Innovation

• Model-Agnostic Calibration
• Martingale PAC-Bayes
• Distribution Shift
• Online Forecasting
• Gated Residual Bayesian Head