Learning-Based Stochastic Optimal Control with Infinite-Horizon Probabilistic Constraints

2026-08-02

AI Summary

This work addresses the stochastic optimal control problem with joint chance constraints over an infinite horizon. By augmenting the state space, the problem is reformulated as a constrained Markov decision process with an additive structure. The paper establishes strong duality for this setting for the first time, thereby equivalently transforming the original problem into an unconstrained Lagrangian dual problem. Building on this duality result, the authors propose a hybrid solution framework that integrates dual ascent with offline value function approximation. This approach significantly reduces online computational complexity while preserving both optimality and probabilistic feasibility of the solution. Numerical experiments demonstrate that, compared to existing online model predictive control strategies, the proposed method achieves comparable control performance with substantially improved computational efficiency.

Abstract

In this paper, we consider stochastic optimal control problems with infinite-horizon joint chance constraints. By means of an appropriate state augmentation, we reformulate the original problem as a constrained Markov decision process, in which both the cost and the constraint function exhibit an additive structure. We then prove that this formulation enjoys strong duality, thereby enabling us to reformulate the problem as an equivalent unconstrained one in the Lagrange dual framework. We propose a dual-ascent algorithm to solve the resulting problem and show that it converges to a deterministic Markov policy defined over the augmented state space that is both optimal and feasible. To accommodate continuous state-input spaces, we propose a dedicated learning algorithm to approximate the value function in an offline training setting, thereby significantly reducing the computational complexity of the online control phase. We then test our approach on a numerical example and demonstrate its effectiveness compared to online predictive control methods in terms of performance and computational complexity.

Problem

• stochastic optimal control
• infinite-horizon
• chance constraints
• Markov decision process
• probabilistic constraints

Innovation

• stochastic optimal control
• infinite-horizon chance constraints
• constrained Markov decision process
• strong duality
• learning-based value approximation