🤖 AI Summary
This work addresses the limitation of evaluating agents solely through reward metrics—such as revenue—which often overlooks strategic discipline, particularly in environments with hidden competitor states. To remedy this, the authors propose a “discipline stability” evaluation paradigm that comprehensively assesses behavioral alignment by defining baseline behaviors, constraining observations, conducting trajectory diagnostics, and performing ablation and transfer experiments. A new benchmark task is introduced, featuring dual-hotel pricing with hidden budget-based bidding, within a multi-agent reinforcement learning framework. The study analyzes trajectory alignment using PPO variants, behavioral cloning, Trace-Prior, and history correction strategies. Results show that pure behavioral cloning achieves near-ideal alignment under symmetric conditions, while Trace-Prior RL enables bounded adaptation under capacity asymmetry; furthermore, hidden states reduce label uncertainty, and reward-only optimization fails to ensure trajectory consistency.
📝 Abstract
Outcome-only evaluation can certify economically unsafe agents: a policy can hit a business KPI while violating deployable behavioral discipline. In hotel pricing with hidden competitor state, a learner can achieve plausible revenue per available room while failing to preserve the rate discipline of a rule-based revenue-management competitor.
We introduce discipline stability, a trace-based evaluation paradigm: define the benchmark behavior, restrict observations to the deployment regime, induce trace diagnostics from failure, separate mechanisms with ablations, and test transfer and deployment. Across a two-hotel benchmark and a compact hidden-budget bidding task, reward-only PPO variants miss trace alignment; revealing hidden state reduces label uncertainty; deterministic copy collapses uncertainty; and trace-prior or corrected history policies better preserve price or bid distributions. Pure behavior cloning is nearly enough for symmetric imitation, while Trace-Prior RL adds bounded adaptation under capacity asymmetry. The contribution is an evaluation and benchmark paradigm, not a new optimizer or a universal claim about MARL