🤖 AI Summary
Reinforcement learning (RL) models suffer performance degradation in non-stationary environments due to shifts in transition dynamics or reward functions. Method: This paper proposes a model drift detection framework grounded in behavioral sequence distribution shifts. It introduces an edit-operation-based distance metric to quantify structural deviations in state-action trajectories, integrated with noise-robust trajectory similarity analysis and statistical distribution comparison for high-sensitivity detection of subtle, incremental drifts. Contribution/Results: Experiments across diverse non-stationary RL benchmarks demonstrate that the method significantly outperforms baselines, accurately distinguishes drift from non-drift episodes, and exhibits strong robustness to observational noise. The framework yields a deployable, interpretable online drift monitoring tool for real-world RL applications—particularly in safety-critical domains such as healthcare, finance, and robotics—where adaptive model maintenance is essential.
📝 Abstract
Reinforcement learning (RL) agents typically assume stationary environment dynamics. Yet in real-world applications such as healthcare, robotics, and finance, transition probabilities or reward functions may evolve, leading to model drift. This paper proposes a novel framework to detect such drifts by analyzing the distributional changes in sequences of agent behavior. Specifically, we introduce a suite of edit operation-based measures to quantify deviations between state-action trajectories generated under stationary and perturbed conditions. Our experiments demonstrate that these measures can effectively distinguish drifted from non-drifted scenarios, even under varying levels of noise, providing a practical tool for drift detection in non-stationary RL environments.