π€ AI Summary
This work addresses the limitation of existing generalization bounds, which assume identical training and test distributions and thus fail to account for distributional shifts arising from varying proportions of calm and crisis regimes in environments with mechanism switches. Focusing on Markov-switching data distributions, the paper proposes a theoretical framework that quantifies the excess risk induced by mismatched regime compositions. The key contributions include extending generalization bounds to Ξ²-mixing sequences, introducing an effective sample size corrected by the spectral gap, and precisely decomposing the effects of regime mismatch and regime sensitivity. A matching minimax lower bound is also established. Empirical results on both synthetic data and 25 years of global equity indices demonstrate that the proposed penalty term significantly predicts ex post generalization gaps, whereas conventional estimators relying solely on the training set lack such predictive power.
π Abstract
The standard generalization bounds assume that the training and deployment distributions are the same, or are static, and don't consider regime switching environments where the ratio of calm vs crisis states is different. This paper proposes a framework that generalizes regime-aware models by quantifying the extra risk due to regime composition mismatch, when distribution shifts are Markov-switching. We obtain an exact decomposition, separating regime mismatch from regime sensitivity; we extend the bound to beta-mixing data using the effective sample size corrected for the spectral gap; and we show a minimax lower bound for synthetic data and on 25 years of global equity indices. The proposed penalty is an ex post realized generalization gap, whereas the training-only estimator does not show significant correlation: the feature geometry of crises can be detected, but not the temporal arrival. Thus, the framework is not a forecast machine. Forecasting the composition of the future regime is an open question in the rare cases of regime change.