model predictive control

To address the challenge of simultaneously ensuring real-time performance, safety, global closed-loop stability, and hard constraint satisfaction in Model Predictive Control (MPC), this paper proposes a deep neural network-based approximation of the infinite-horizon value function. The network is trained via joint value iteration and nonlinear trajectory optimization to learn a constraint-admissible optimal terminal cost. This work presents the first provably globally stable neural parameterization of an infinite-horizon value function for MPC; when embedded online, it eliminates the need for manual terminal cost design while inherently guaranteeing closed-loop stability. Evaluated on standard simulation benchmarks and a real-world industrial robotic arm performing online obstacle avoidance, the method significantly improves constraint satisfaction rates and closed-loop stability, while enabling complex real-time tasks such as dynamic obstacle evasion.

This work investigates how to achieve optimal policies in Markov decision processes (MDPs) that incorporate future information—such as reference trajectories or predictions—by leveraging model predictive control (MPC). The authors formulate MPC as a class of parameterized policies and train them end-to-end via reinforcement learning. Their key contribution lies in establishing, for the first time, the precise structural conditions under which MPC can exactly represent the optimal value function and policy, thereby providing a theoretical foundation for MPC as a structured function approximator with formal guarantees. Empirical validation on a point-mass racing task with future reference trajectories demonstrates that the proposed approach learns policies approaching optimality, confirming its effectiveness.

This work addresses the problem of learning non-stationary optimal policies for finite-horizon constrained Markov decision processes (CMDPs), filling a theoretical gap left by prior studies focused on infinite-horizon settings and stationary policies. We propose the first policy gradient algorithm specifically designed for finite-horizon CMDPs, employing time-varying parametric policy networks and Lagrangian relaxation to handle hard constraints. We provide rigorous convergence guarantees—proving that the algorithm converges to a constraint-optimal solution under standard regularity conditions. The method accommodates continuous state-action spaces, supports function approximation, and scales effectively to high-dimensional problems. Empirical evaluation across multiple benchmark tasks demonstrates substantial improvements in cumulative reward, constraint satisfaction rate, and convergence stability, thereby validating both the theoretical assurances and practical efficacy of the approach.

This paper investigates the suboptimality of nominal model-predictive linear-quadratic (LQ) control for unknown linear systems, characterizing a fundamental trade-off among model mismatch, terminal cost approximation error, and prediction horizon length. We develop a novel perturbation analysis framework for the Riccati difference equation, establishing—for the first time—a quantitative relationship between horizon length and the system’s controllability index. Theoretically, we prove that a finite horizon bounded by the controllability index suffices to approximate infinite-horizon optimal performance, and that horizons of length one or infinity are often optimal. Based on this insight, we derive the first adaptive horizon-selection criterion tailored for learning-based control, yielding a tight suboptimality upper bound, an $O(log T)$ regret guarantee, and optimal sample complexity.

This work proposes Drifting MPC, a novel framework for offline reinforcement learning in settings where the system dynamics are unknown and trajectory simulation is infeasible. Drifting MPC uniquely integrates a drift generative model with model predictive control to learn a conditional trajectory distribution from offline data that balances data support and cost optimality. The method explicitly optimizes task-specific costs while maintaining fidelity to the empirical data distribution, and it is theoretically shown that the resulting distribution constitutes the unique solution that optimally trades off optimality against consistency with the data prior. Empirical results demonstrate that Drifting MPC efficiently generates near-optimal trajectories with low per-step computational overhead, significantly reducing trajectory generation time compared to diffusion-model baselines.

This work addresses the challenge of real-time robotic arm control in dynamically cluttered environments, where agents must balance rapid responsiveness with foresightful obstacle avoidance to prevent myopic constraint violations. The authors propose a task-space receding horizon controller that generates collision-free terminal pose references through short-horizon, contact-consistent forward simulations respecting non-penetration constraints, then computes only the first-step minimum-acceleration control input that smoothly transitions toward this reference. By integrating the strengths of receding horizon and reactive control, the method efficiently embeds information about contacts, moving obstacles, and self-collisions using inflated convex geometry and an iterative dynamics solver—without requiring full trajectory optimization. Simulations with 40 degrees of freedom demonstrate that a moderate horizon length effectively balances foresight, responsiveness, and computational cost, while hardware experiments on a 6-DOF manipulator confirm strong sim-to-real transfer, outperforming MPC and dynamic optimization fabric approaches in success rate under dynamic clutter while meeting real-time requirements.

This work addresses the challenge of navigating complex cost landscapes in non-convex model predictive control, where nonlinear dynamics and multiple obstacles often trap gradient-based methods in suboptimal local minima. To overcome this limitation, we propose a Maximum Entropy Differential Dynamic Programming (ME-DDP) framework that integrates deterministic optimization with entropy-maximizing sampling. Our approach employs a two-stage mechanism: it first performs local gradient-based refinement via DDP and then leverages the inverse Hessian of the action-value function to guide policy sampling, enabling escape from local minima and balancing global exploration with local exploitation. We develop three ME-DDP variants, elucidate their theoretical connections to Model Predictive Path Integral (MPPI) control, and demonstrate superior performance across four navigation benchmarks—achieving higher success rates in high-dimensional systems, outperforming MPPI in low-dimensional settings, and exhibiting robustness in real-world quadrotor experiments through dense obstacle fields.

This work addresses the stochastic optimal control problem with joint chance constraints over an infinite horizon. By augmenting the state space, the problem is reformulated as a constrained Markov decision process with an additive structure. The paper establishes strong duality for this setting for the first time, thereby equivalently transforming the original problem into an unconstrained Lagrangian dual problem. Building on this duality result, the authors propose a hybrid solution framework that integrates dual ascent with offline value function approximation. This approach significantly reduces online computational complexity while preserving both optimality and probabilistic feasibility of the solution. Numerical experiments demonstrate that, compared to existing online model predictive control strategies, the proposed method achieves comparable control performance with substantially improved computational efficiency.