rollout analysis

To address model drift caused by dynamic evolution of real-world data distributions, this paper proposes an interpretable drift detection method. Unlike existing black-box or weakly interpretable approaches, our method introduces the first risk-aware hypothesis testing framework that explicitly accounts for feature interactions. It unifies drift detection and root-cause interpretation from two perspectives: statistical power and task generality—supporting both classification and regression. By modeling high-order feature interactions, incorporating risk-sensitive test statistics, quantifying interpretability, and localizing drift-sensitive features, the method achieves superior performance over state-of-the-art interpretable methods on multiple benchmark drift datasets and real-world scenarios, while matching the accuracy of leading black-box approaches. Case studies further validate its capability to precisely identify underlying drift mechanisms and critical driving features.

This study addresses the critical issue of model instability in software engineering optimization, which leads to substantial variability across repeated experiments and undermines both credibility and practical utility. Rather than treating instability as mere random noise, this work conceptualizes it as a quantifiable and manageable property that should be integrated into standard evaluation frameworks. By systematically modulating label usage, model complexity, and partition scoring strategies—combined with multi-objective optimization, causal intervention, data locality analysis, and model calibration—the proposed approach significantly enhances result consistency. Empirical evaluation demonstrates that the optimized configuration reduces the standard deviation of error by 22% on average and outperforms default settings in 119 out of 127 datasets, achieving a 4.8-fold improvement in result consistency.

Reinforcement learning (RL) models suffer performance degradation in non-stationary environments due to shifts in transition dynamics or reward functions. Method: This paper proposes a model drift detection framework grounded in behavioral sequence distribution shifts. It introduces an edit-operation-based distance metric to quantify structural deviations in state-action trajectories, integrated with noise-robust trajectory similarity analysis and statistical distribution comparison for high-sensitivity detection of subtle, incremental drifts. Contribution/Results: Experiments across diverse non-stationary RL benchmarks demonstrate that the method significantly outperforms baselines, accurately distinguishes drift from non-drift episodes, and exhibits strong robustness to observational noise. The framework yields a deployable, interpretable online drift monitoring tool for real-world RL applications—particularly in safety-critical domains such as healthcare, finance, and robotics—where adaptive model maintenance is essential.

This study addresses the performance degradation of portfolio rebalancing strategies under distributional shifts caused by regime changes in market mechanisms. To mitigate this issue, the authors propose the Scenario Context Unfolding (SCR) method, which generates multivariate return scenarios under stress conditions conditioned on macroeconomic states and constructs counterfactual next-state representations to correct the reward-transition mismatch in temporal difference learning. This approach stabilizes the training of reinforcement learning critics by enabling a controllable bias-variance trade-off under distributional shift. Evaluated on a testbed of 31 U.S. equity and ETF portfolios, SCR significantly enhances robustness, yielding up to a 76% improvement in Sharpe ratio and a reduction in maximum drawdown of up to 53%.

This work investigates the impact of dynamics and reward model errors on policy performance in imagination-based trajectory learning. By extending MDP error analysis to settings with learned reward models, the authors reformulate policy optimization under reward noise as a one-dimensional problem, targeting representations with low Lipschitz constants. Integrating theoretical error bounds, REINFORCE gradient estimation, and sample complexity analysis, they derive the optimal allocation ratio between dynamics and reward samples under a fixed sampling budget. The analysis theoretically establishes that zero-mean reward noise introduces no bias and that its variance decays with the number of imagined trajectories, yielding clear practical guidelines for sample-efficient policy training.

This work addresses the challenge of silent updates to large language models (LLMs) by service providers, which often occur without version changes and can lead to behavioral drift and functional regressions, while existing mechanisms lack deployment-side control over compatibility governance. Framing LLM updates as a software supply chain governance problem, this study proposes a deployment-side control framework that defines rule-based production contracts, constructs risk-category-oriented test suites, and enforces compatibility gates to validate model safety and performance prior to updates. Experimental results demonstrate that the approach effectively uncovers fine-grained regressions missed by aggregate metrics, while also highlighting critical challenges in test design, threshold calibration, and drift attribution.

This work addresses the limitations of existing instruction data filtering methods, which struggle to identify the most beneficial samples for model improvement due to reliance on external validation targets and bias in gradient norms. The authors propose DRIFT, a novel approach that leverages on-policy generated outputs as validation targets within an influence function framework. By incorporating trajectory correctness–based sign weighting and gradient bias correction, DRIFT enables precise sample selection and reweighting for supervised fine-tuning. This strategy effectively reduces the parameter proximity gap and refines the training data distribution. Evaluated on 7B-scale instruction-following and reasoning models, DRIFT significantly outperforms current data filtering baselines and pushes the performance ceiling of fine-tuned models.

This work proposes a policy repair evaluation paradigm that operates without access to expert action labels or reward signals at the state level. Focusing on a hotel pricing simulation environment, the approach leverages only region-level diagnostic feedback—summarizing discrepancies between the current and baseline policies across temporal, inventory, and market dimensions of price distributions. A multi-agent LLM architecture integrates diagnostic projection, a tree editor, and a non-semantic proposer to perform constrained policy editing. Evaluated over 5,000 test episodes, the method achieves a RevPAR of 108.47 (approaching the baseline’s 108.75) and significantly reduces episode composition distance to 0.609, outperforming multiple baselines. The study further reveals that aggregate alignment can mislead repair outcomes, underscoring the necessity of translating diagnostic feedback into reliable closed-loop evaluation metrics.

This study addresses the performance degradation of machine learning classifiers in dynamic environments caused by concept drift, a phenomenon inadequately captured by conventional evaluation methods that overlook causal dependencies in data, leading to distorted assessments. To overcome this limitation, the authors propose a digital twin framework grounded in Structural Causal Models (SCMs), which, for the first time, leverages SCMs to simulate realistic causal drift. By applying parametric causal interventions, the framework stress-tests classifiers while preserving the underlying structure of the data-generating mechanism. This approach transcends the constraints of traditional statistical or correlation-based evaluations. Experimental results on the OSMH dataset demonstrate that the method effectively uncovers classifier vulnerabilities that remain undetected by standard monitoring techniques.

In policy-based reinforcement learning, redundant transitions arising from causal dependencies within rollouts can lead to gradient duplication, value network lag, and training instability. This work proposes a simple yet effective sampling strategy that randomly discards a fixed proportion (e.g., 25%) of transition samples while preserving the full reward signal, thereby breaking redundant structures and stabilizing PPO training. The method requires only a single additional sampling step and does not alter the core PPO algorithm or introduce new components. Experimental results across five standard environments demonstrate that this approach significantly improves training consistency—measured by KL divergence, policy entropy, and value estimation—while maintaining comparable returns.