RL-PaO: Prediction as Action in Decision Making under Uncertainty

2026-09-29

AI Summary

This study addresses the disconnect between predictive accuracy and downstream decision quality by proposing a reinforcement learning-based end-to-end optimization framework. The proposed approach formulates prediction as an action, unifying system modeling, optimization, and execution within a Markov Decision Process. By directly interacting with the environment, it aligns prediction errors with actual decision costs without requiring gradient computation through black-box solvers. Applied to day-ahead energy scheduling tasks, the method achieves an average 10% cost reduction while outperforming non-ideal baselines, all while maintaining high interpretability.

Abstract

Decision-making under uncertainty often relies on predicted parameters, yet accurate prediction does not necessarily lead to good operational decisions. Aligning prediction with downstream optimization requires learning from the consequences of the decisions those predictions induce. We introduce RL-PaO, a reinforcement learning framework that integrates system formulation, optimization, and decision execution into a single environment. This yields a Markov decision process in which prediction is regarded as action: it shifts the environment to produce subsequent context and reward that explicitly aligns prediction error with realized cost, and learning the optimal policy does not require differentiating through the black-box solver. We evaluate RL-PaO on day-ahead energy scheduling using real historical data. On the test year, RL-PaO achieves the lowest annual cost among the non-oracle baselines, achieving on average $10\%$ cost reduction. Moreover, RL-PaO is capable of further analyses to provide strong interpretability both from the policy evolution perspective and the cost-accuracy trade-off.

Problem

• decision-making under uncertainty
• prediction
• optimization
• reinforcement learning
• energy scheduling

Innovation

• Reinforcement Learning
• Prediction as Action
• Decision Making under Uncertainty
• Black-box Optimization
• Interpretability