Adaptive Regime-Switching Forecasts with Distribution-Free Uncertainty: Deep Switching State-Space Models Meet Conformal Prediction

2025-12-02

AI Summary

This paper addresses the challenge of calibrating prediction uncertainty in nonstationary time series induced by regime shifts. We propose a distribution-free, online-updatable uncertainty quantification method. Methodologically, we (1) develop a deep switching state-space model to capture dynamic nonstationarity; (2) design a unified conformal wrapping framework that integrates adaptive conformal inference (ACI) and aggregated ACI (AgACI), providing finite-sample marginal coverage guarantees under model misspecification and limited data; and (3) ensure compatibility with diverse sequence models—including S4, MC-Dropout GRU, and sparse Gaussian processes. Experiments on synthetic and real-world benchmarks demonstrate that our approach achieves coverage probabilities close to the nominal level, maintains competitive point prediction accuracy, and substantially narrows prediction intervals compared to existing methods.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Intelligent Robots: State Estimation

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Regime transitions routinely break stationarity in time series, making calibrated uncertainty as important as point accuracy. We study distribution-free uncertainty for regime-switching forecasting by coupling Deep Switching State Space Models with Adaptive Conformal Inference (ACI) and its aggregated variant (AgACI). We also introduce a unified conformal wrapper that sits atop strong sequence baselines including S4, MC-Dropout GRU, sparse Gaussian processes, and a change-point local model to produce online predictive bands with finite-sample marginal guarantees under nonstationarity and model misspecification. Across synthetic and real datasets, conformalized forecasters achieve near-nominal coverage with competitive accuracy and generally improved band efficiency.

Problem

• Addresses regime-switching time series forecasting with distribution-free uncertainty quantification.
• Combines deep state-space models and conformal inference for online predictive bands.
• Ensures marginal coverage guarantees under nonstationarity and model misspecification.

Innovation

• Deep Switching State Space Models with Adaptive Conformal Inference
• Unified conformal wrapper for nonstationary time series
• Online predictive bands with finite-sample marginal guarantees