value prediction

This paper addresses two-stage stochastic optimization problems with contextual information. Method: We propose a novel “single-scenario optimal solving” paradigm: under fixed recourse matrices and linear second-stage costs, we theoretically establish for the first time that such problems reduce to point-estimate optimization over a single scenario. We develop a joint learning-and-optimization framework featuring a decision-optimal structured loss function, which trains a parametric forecasting model to produce point predictions explicitly tailored to optimal decisions. Contribution/Results: On synthetic inventory control and real-world bike-sharing dispatch tasks, our approach reduces decision cost by 12–23% compared to conventional “predict-then-optimize” pipelines and distributional forecasting baselines, while cutting computational overhead by an order of magnitude—significantly enhancing end-to-end decision-making efficacy.

This work addresses a critical limitation in existing latent-variable world models, which rely on average prediction error over training data for training and selection—a metric that fails to reflect actual controller performance due to a mismatch between the evaluation distribution and the distribution queried by the planner. The authors propose instead to center model assessment on the discrepancy between predicted and true costs over states reachable by the planner. They establish, for the first time, a rigorous theoretical link between this discrepancy and control suboptimality, proving it provides a valid upper bound on performance loss, whereas conventional prediction errors neither bound nor track performance. Leveraging control theory, spectral analysis, and non-normal operator theory, they decompose the discrepancy into an intrinsic manifold residual and an off-manifold divergence term, and introduce a fidelity score to quantify alignment of the planner’s reachable distribution. Experiments on synthetic systems and model predictive control confirm that the proposed metric reliably tracks control performance, while single-step prediction error shows virtually no correlation.

Addressing the dual challenges of inherent stochasticity and non-differentiable evaluation metrics in physical spatiotemporal forecasting, this paper proposes a novel model-based reinforcement learning paradigm that reformulates prediction as sequential planning. Methodologically, we construct a generative world model to simulate high-fidelity, diverse future states and employ domain-specific non-differentiable metrics—such as extreme-event hit rate—as sparse reward signals. We design a beam-search–guided, reward-driven imagination mechanism and introduce an iterative pseudo-labeling self-training strategy. Crucially, our framework enables end-to-end optimization of non-differentiable objectives without gradient approximation. Experiments demonstrate substantial reductions in overall prediction error alongside marked improvements in long-tail event detection. This work establishes a new pathway toward interpretable and robust forecasting for complex physical systems.

This work investigates how to achieve optimal policies in Markov decision processes (MDPs) that incorporate future information—such as reference trajectories or predictions—by leveraging model predictive control (MPC). The authors formulate MPC as a class of parameterized policies and train them end-to-end via reinforcement learning. Their key contribution lies in establishing, for the first time, the precise structural conditions under which MPC can exactly represent the optimal value function and policy, thereby providing a theoretical foundation for MPC as a structured function approximator with formal guarantees. Empirical validation on a point-mass racing task with future reference trajectories demonstrates that the proposed approach learns policies approaching optimality, confirming its effectiveness.

This work addresses cross-task generalization in reinforcement learning, where agents must rapidly adapt to unseen tasks that are dynamically similar yet differ in reward functions. To this end, we propose Task-Aware Dreamer (TAD), a novel framework introducing (i) a reward-aware world model and (ii) a task-discriminative variational objective, alongside the Task Distribution Relevance (TDR) metric to quantify inter-task divergence. TAD integrates a variational inference-based world model, reward-conditioned latent representation learning, Dreamer-style model-based prediction and policy optimization, and a TDR-driven policy selection mechanism. Experiments across image- and state-space multi-task benchmarks demonstrate substantial improvements in parallel training efficiency and zero-shot generalization performance. Notably, TAD significantly outperforms conventional Markovian policies—especially under high-TDR conditions—highlighting its efficacy in handling reward-divergent task distributions.

This work addresses the challenge that large language model agents struggle with effective forward-looking planning in long-horizon tasks due to the absence of an internalized world model. To overcome this limitation, the authors propose a capability-first, three-stage training framework comprising World-Model Agent Mid-training (WM-AMT), Format-Guided Supervised Fine-Tuning (FE-SFT), and Foresight-Conditioned Reinforcement Learning (FC-RL). This framework enables a single autoregressive model to generate coherent future state rollouts and calibrate the likelihood of plan success. Evaluated on search and mathematical reasoning tasks, the approach significantly outperforms existing baselines, achieving— for the first time—a well-calibrated internal world model with genuine predictive capacity, thereby effectively bridging the gap between output format alignment and planning competence.

This work addresses the absence of a unified closed-loop learning environment that enables agents to continuously learn from real-world events and forecast future outcomes. To bridge this gap, we propose FutureWorld—the first framework that formulates real-time future prediction as a reinforcement learning environment. By integrating a closed-loop mechanism of prediction, outcome realization, and parameter update, FutureWorld effectively prevents answer leakage and supports continual learning. Built upon open-source large language models and grounded in real-world event feedback, the framework establishes a daily-updated benchmark for training and evaluation. Experimental results over consecutive days demonstrate the efficacy of our approach, setting a new state-of-the-art baseline and significantly advancing agents’ predictive capabilities.

This work addresses the high computational cost and limited generalization of traditional state-value-based approaches in planning tasks by proposing a novel supervised learning framework that directly trains a regularized Q-function. The method models the Q-function using a graph neural network and introduces an action-discriminative regularization term to enhance the distinction between actions selected by the teacher policy and those not selected. To our knowledge, this is the first systematic effort in planning-by-learning to replace state-value functions with Q-functions. Evaluated across ten planning domains, the approach significantly outperforms existing state-value-based policies, achieving performance on par with the state-of-the-art planner LAMA-first while substantially reducing per-step inference cost, thereby improving both inference efficiency and policy robustness.

Current evaluation standards for world models suffer from inconsistency and a lack of task-specific focus, particularly in embodied decision-making scenarios, often leading to claims unsupported by empirical evidence. This work proposes a decision-centric evaluation framework featuring a novel L0–L7 assessment hierarchy that clearly distinguishes diagnostic metrics from those measuring decision utility. Emphasizing core capabilities such as counterfactual reasoning, closed-loop rollouts, and policy optimization, the framework introduces a new benchmark protocol centered on action fidelity, policy ranking consistency, and uncertainty calibration. This protocol is operationalized through counterfactual intervention analysis, closed-loop policy rollouts, value prediction tasks, and model exploitability tests, systematically addressing critical gaps in existing evaluation practices and establishing clear, reproducible validation criteria for world model research.

This work addresses the limited generalization of reinforcement learning agents to new tasks, which often necessitates training from scratch. To overcome this, the authors propose Outcome-Predictive State Representations (OPSRs) and the OPSR Skill framework, which construct compact, task-agnostic state abstractions and define reusable abstract actions—referred to as skills—on top of these representations. This approach is the first to jointly abstract both states and actions, enabling cross-task skill transfer without requiring task-specific preprocessing, while preserving policy optimality. Empirical results demonstrate that OPSR-based skills significantly accelerate learning across multiple unseen tasks, confirming their strong generalization capability and effectiveness.