Exposure Bias as Epistemic Underidentification in Recursive Forecasting

2026-06-11

AI Summary

This work addresses the distributional mismatch—commonly termed exposure bias—in recursive multi-step prediction, which arises from state truncation or partial observability and cannot be fully explained by conventional distribution shift theories. The authors reframe this issue as a self-induced epistemic uncertainty problem and introduce an analytical framework centered on “induced states” \( Z \) and “provenance variables” \( P \), formally characterizing the model’s objective uncertainty under self-generated states and decomposing error sources. Their approach integrates Bayesian supervised modeling, closed-loop correction, and binary provenance encoding. Experiments reveal that recursive inference enters distinct regions of induced states, where local correction mechanisms synergize with induced states to enhance performance. Furthermore, provenance-aware correction proves effective, though its benefits are contingent upon specific conditions.

Abstract

Recursive multi-step forecasting is usually framed as distribution shift: models are trained on observed histories but deployed on their own predictions. We show this framing is incomplete by proving that, under partial observability or state truncation, recursive rollout is also an epistemic underidentification problem. Even with deterministic latent dynamics, one-step Bayes supervision identifies behavior only on observed contexts and need not identify the deployed recursive predictor once rollout queries self-generated induced states whose correct local targets are not determined by numeric state alone. We formalize this with induced states $Z$ and provenance variables $P$, and derive a decomposition of induced-state error into teacher-forcing/rollout mismatch, representation--class approximation, and provenance information gaps. Empirically, we show that rollout enters a distinct induced-state regime, that fixed induced states define a distinct local corrective task, and that closed-loop gains arise not only from local adaptation but also from changing the induced states visited during rollout. Using a simple binary provenance encoding, provenance-aware correction can further improve performance, though gains are conditional rather than uniform. These results recast exposure bias as reasoning under self-induced epistemic uncertainty.

Problem

• exposure bias
• epistemic underidentification
• recursive forecasting
• induced states
• partial observability

Innovation

• epistemic underidentification
• recursive forecasting
• induced states
• provenance variables
• exposure bias