design control policies

Addressing the dual challenges of theoretical guarantee gaps and empirical conservativeness in robust control of linear systems under model mismatch, this paper introduces conformal prediction into the LQR framework for the first time. It proposes a data-driven, distribution-free uncertainty modeling approach that provides statistically valid coverage guarantees for true system dynamics. Theoretically, it establishes a probabilistic characterization of regret—breaking away from deterministic robustness bounds in classical robust control. Algorithmically, it designs a novel policy gradient optimizer with provable convergence guarantees. Experiments on multiple engineering control systems demonstrate that the method achieves a superior trade-off between closed-loop performance and conservatism, significantly outperforming both H∞ control and multiplicative-noise LQR.

Traditional robot design and control are typically decoupled, leading to morphologies poorly aligned with task requirements. This paper proposes a simulation-driven co-optimization framework for morphology and control, breaking the conventional “design-then-control” paradigm to enable task-oriented, end-to-end joint search. Our method employs gradient-free optimization to simultaneously evolve structural parameters and controller policies within a URDF-based multi-task reinforcement learning simulation environment. Key contributions include: (1) demonstrating that controller retraining significantly improves performance, yielding an average gain of 37%; and (2) revealing an inverse correlation between morphological complexity and controller training budget—providing theoretical justification for structural simplification under resource constraints. We validate the framework across four public simulation benchmarks, showing that co-optimization consistently yields more compact, robust, and task-adapted robot morphologies compared to sequential approaches.

Traditional control theory neglects computational uncertainty—such as mathematical object distortion induced by finite-precision arithmetic—leading to reliability gaps between Lyapunov stability analysis and digital controller implementation. Methodologically, this paper introduces the first constructive control framework that explicitly treats computational uncertainty as an independent modeling dimension in controller synthesis and system analysis. Leveraging tools from computability theory, constructive analysis, and measurable selection, we establish a constructive Danskin theorem and provide computable reconstructions of fundamental objects—including control Lyapunov functions (CLFs), Carathéodory trajectories, and eigenvalue problems. Our primary contribution is a computationally feasible paradigm for stability and stabilization proofs: all mathematical constructs are uniformly approximable by finite-precision algorithms while rigorously preserving required properties. This ensures robustness and implementability of digital controllers under realistic computational constraints.

This work addresses discrete-time nonlinear optimal control problems by unifying classical algorithms—including gradient descent, Gauss–Newton, Newton’s method, and differential dynamic programming (DDP)—within a differentiable programming framework. Methodologically, it introduces the first modular, end-to-end differentiable algorithm template library built upon linear/quadratic approximations (e.g., LQR), enabled by automatic differentiation. Theoretically, it provides a unified derivation of computational complexity and sufficient optimality conditions across all methods. Practically, it incorporates adaptive line search and regularization strategies, and validates efficacy on benchmark tasks such as autonomous racing with a bicycle model. All implementations are open-sourced, demonstrating both efficient gradient propagation and strong generalization across diverse control problems.

This paper addresses the infinite-horizon optimal closed-loop control problem for nonlinear systems with unknown dynamics, aiming to minimize a given cost function from arbitrary initial states without relying on an explicit system model. We propose a data-driven policy optimization method that integrates the Koopman operator with an actor-critic framework: the Koopman operator enables model-free dynamical representation and differentiable cost gradient estimation, while a parameterized policy is updated via stochastic gradient descent. To our knowledge, this is the first model-free policy gradient method with theoretically guaranteed convergence. Experiments demonstrate stable convergence across multiple nonlinear systems, with control performance significantly surpassing standard model-free reinforcement learning algorithms and closely approaching the optimal benchmark achievable under full model knowledge.

This work addresses the problem of feedback motion planning for continuous-time stochastic nonlinear systems under Signal Temporal Logic (STL) specifications by proposing a novel framework that integrates predicate erosion with probabilistic reachable tubes. Predicate erosion is employed to transform stochastic STL constraints into tightened deterministic ones, while probabilistic reachable tubes quantify the deviation of stochastic trajectories from their nominal counterparts. Leveraging contraction theory, a tracking controller is designed to establish a closed-loop planning pipeline. The proposed approach significantly reduces the conservatism inherent in conventional methods, achieving high STL satisfaction probability without compromising planning performance. Simulations and real-world experiments on a quadrupedal robot demonstrate that the method outperforms baseline approaches in both STL satisfaction rate and computational efficiency.

This study addresses robust hierarchical control and topology co-design for interconnected linear network systems when precise subsystem models are unavailable and only data are accessible, ensuring dissipativity from disturbances to performance outputs. Two strategies are proposed: a model-driven approach integrating local dissipativity-based control with global topology optimization, and a data-driven method that relies solely on input–state–output trajectories, leveraging the matrix S-lemma under quadratic constraints on disturbances to jointly design control and topology. The work innovatively unifies dissipativity theory with topology optimization, enabling composable and decentralized co-design while relaxing disturbance assumptions and circumventing centralized nonconvex iterations through a model-free framework. The efficacy of the approach is validated in a DC microgrid, demonstrating improved voltage regulation, current sharing, and optimized interconnection costs.

This work addresses the challenge of effectively translating social preferences into resource allocation objectives within multi-agent control systems to fulfill ethical and socially responsible missions. By aggregating individual preferences into a welfare-oriented control objective, the study unifies this approach across three major control paradigms: online feedback optimization, Markov decision process control, and model predictive control. It presents the first systematic framework that embeds social welfare principles directly into the control design pipeline, integrating preference aggregation with formal verification mechanisms to yield a certifiably compliant control architecture. This framework offers a novel pathway for automated resource allocation systems that simultaneously ensures fairness, efficiency, and interpretability.

This study addresses the lack of a systematic synthesis in research on integrating reinforcement learning (RL) with model predictive control (MPC) for linear systems by proposing the first multidimensional taxonomy tailored to this domain. Drawing on a comprehensive literature review up to 2025, the work establishes a classification framework along five dimensions: RL role, algorithm type, MPC formulation, cost function structure, and application area, followed by an integrative cross-dimensional analysis. The study elucidates representative integration strategies, traces methodological evolution, and identifies key challenges—including computational burden, sample efficiency, robustness, and closed-loop guarantees—thereby offering a structured reference and practical guidance for both theoretical analysis and architectural design in RL–MPC systems.