React to Surprises: Stable-by-Design Neural Feedback Control and the Youla-REN

2025-06-02

AI Summary

This work addresses the problem of stabilizing nonlinear, partially observable systems via neural control. We propose an end-to-end learning framework intrinsically guaranteeing incremental stability. Methodologically, we integrate nonlinear Youla–Kučera parameterization with recurrent equilibrium networks (RENs), establishing—for the first time—a joint d-tube contraction and Lipschitz continuity certification mechanism. This enables a complete, unconstrained parameterization of all contraction-Lipschitz closed-loop controllers. Our key contributions are: (1) overcoming the triply coupled challenges of nonlinear dynamics, partial observability, and incremental stability (i.e., contraction plus Lipschitz robustness); (2) ensuring weak yet practically meaningful closed-loop stability *naturally*, without explicit stability constraints during optimization; and (3) demonstrating—through experiments—that the method efficiently learns controllers with rigorous stability certificates under low-sample regimes, economic reward settings, and model uncertainty, significantly improving both robustness and convergence.

Technology Category

Intelligent Robots: Behavior Learning & ControlSearch and Optimization: Learning to SearchNatural Language Processing: Learning & Optimization for NLP

Application Category

Economics, Online Markets and Human Computation: LLM based quality controls for crowd workResponsible Web: Machine-in-the-loop, human agency and autonomySearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We study parameterizations of stabilizing nonlinear policies for learning-based control. We propose a structure based on a nonlinear version of the Youla-Kuv{c}era parameterization combined with robust neural networks such as the recurrent equilibrium network (REN). The resulting parameterizations are unconstrained, and hence can be searched over with first-order optimization methods, while always ensuring closed-loop stability by construction. We study the combination of (a) nonlinear dynamics, (b) partial observation, and (c) incremental closed-loop stability requirements (contraction and Lipschitzness). We find that with any two of these three difficulties, a contracting and Lipschitz Youla parameter always leads to contracting and Lipschitz closed loops. However, if all three hold, then incremental stability can be lost with exogenous disturbances. Instead, a weaker condition is maintained, which we call d-tube contraction and Lipschitzness. We further obtain converse results showing that the proposed parameterization covers all contracting and Lipschitz closed loops for certain classes of nonlinear systems. Numerical experiments illustrate the utility of our parameterization when learning controllers with built-in stability certificates for: i) ``economic'' rewards without stabilizing effects; ii) short training horizons; and iii) uncertain systems.

Problem

• Parameterizing stabilizing nonlinear policies for learning-based control
• Ensuring closed-loop stability with unconstrained neural network structures
• Addressing incremental stability challenges in nonlinear dynamics and partial observation

Innovation

• Combines Youla-Kučera with robust neural networks
• Ensures closed-loop stability by construction
• Maintains d-tube contraction under disturbances