Bridging Reinforcement Learning and Optimal Control via Feasible Action Mapping

2026-07-26

AI Summary

This work addresses the performance degradation in existing reinforcement learning approaches for constrained dynamical systems, where infeasible actions often compromise control efficacy. To overcome this limitation, the paper introduces the Feasible Action Optimal Control (FAOC) framework, which employs an optimization-based action-mapping mechanism to project the agent’s abstract actions onto a state-dependent set of feasible control parameters in real time—eliminating the need for handcrafted, expert-designed action spaces. By seamlessly integrating the policy flexibility of reinforcement learning with the rigorous safety guarantees of optimal control, FAOC ensures recursive feasibility and strict constraint satisfaction. Evaluated on a real-time robotic table tennis planning task, the proposed method demonstrates superior sample efficiency and closed-loop control performance compared to state-of-the-art baselines.

Abstract

Operating constrained dynamical systems requires controllers to efficiently solve complex tasks while enforcing recursive feasibility and safety constraints. To address these competing requirements, we present Feasible Action for Optimal Control (FAOC), a novel control framework integrating Reinforcement Learning (RL) and Optimal Control (OC). The key contribution is a computationally efficient, optimization-based mapping algorithm that transforms the RL agent's action from a static abstract set into a state-dependent feasible parameter set of the Optimal Control Problem (OCP), guaranteeing strict satisfaction of the dynamical system's constraints. Thus, FAOC effectively combines the predictable safety of OC with the flexibility of RL. In contrast to prior work, the abstract action space of the RL agent does not require expert or heuristic design, and the OCP formulation is not compromised by the inability of RL to guarantee feasibility. We apply our approach to real-time motion planning for robot table tennis, which encapsulates these challenges. Via simulated experiments, we show that FAOC outperforms state-of-the-art baselines in both sample efficiency and closed-loop performance.

Problem

• constrained dynamical systems
• recursive feasibility
• safety constraints
• real-time motion planning
• optimal control

Innovation

• Feasible Action Mapping
• Reinforcement Learning
• Optimal Control
• Recursive Feasibility
• Constrained Dynamical Systems