🤖 AI Summary
This work proposes a policy repair evaluation paradigm that operates without access to expert action labels or reward signals at the state level. Focusing on a hotel pricing simulation environment, the approach leverages only region-level diagnostic feedback—summarizing discrepancies between the current and baseline policies across temporal, inventory, and market dimensions of price distributions. A multi-agent LLM architecture integrates diagnostic projection, a tree editor, and a non-semantic proposer to perform constrained policy editing. Evaluated over 5,000 test episodes, the method achieves a RevPAR of 108.47 (approaching the baseline’s 108.75) and significantly reduces episode composition distance to 0.609, outperforming multiple baselines. The study further reveals that aggregate alignment can mislead repair outcomes, underscoring the necessity of translating diagnostic feedback into reliable closed-loop evaluation metrics.
📝 Abstract
Agentic AI systems are increasingly used to edit, refine, and repair decision policies, but evaluating these edits is difficult when per-state expert action labels are unavailable. We study this problem in a hotel-pricing simulator where an agentic policy editor receives only region-level diagnostic feedback: summaries of how its price distribution differs from a benchmark policy across time, inventory, and market regions. The editor cannot observe benchmark actions, benchmark source code, reward numbers, or held-out outcomes, and may only propose constrained edits to a target-action table. On 5,000 held-out episodes, a multi-restart LLM editor reaches RevPAR 108.47 (95% CI 107.61 - 109.34), close to the benchmark policy's 108.75 (107.81 - 109.68), with paired gap (LLM minus benchmark) -0.276 and 95% CI [-0.692, 0.146]. A cheap diagnostic projection already recovers much of the revenue (107.90), so the LLM editor's distinctive gain is not raw revenue lift alone: it also reduces episode composition distance from 1.153 to 0.609. This is the strongest non-benchmark repair result. This profile is not explained by restart search alone: non-semantic proposers with up to 2,500 evaluations fall 8.77 - 14.57 RevPAR points short. Nor is it explained by plausible prompt format: a shuffled-diagnostic control breaks region-error correspondence and falls to RevPAR 94.30. The match is genuine but partial. A tree editor achieves stronger pooled alignment, 0.214 versus 0.266, and stronger reference-state D1, 0.328 versus 1.197, yet revenue falls to 98.91. These results show that agentic policy repair should be evaluated by whether diagnostic feedback becomes reliable closed-loop outcome, not by a single behavioral distance.