HypEMBER: Hypernetwork-based Ensemble for Robust Policy Learning of Parametrized Dynamical Systems

2026-07-21

AI Summary

This work addresses the challenge of robust control for parametric dynamical systems under model and measurement uncertainties, where existing reinforcement learning approaches often fail due to high-dimensional states, costly simulations, and difficulties in parameter estimation. To overcome these limitations, we propose a novel framework that integrates conditional hypernetworks with ensemble-based reinforcement learning. The hypernetwork dynamically generates policy and value function weights conditioned on physical parameters, enabling generalization across varying system dynamics. Concurrently, an ensemble of models quantifies epistemic uncertainty to enhance exploration and robustness. Evaluated on the Kuramoto–Sivashinsky equation and a two-dimensional circulation navigation task, our method demonstrates markedly improved training stability and sample efficiency, while achieving superior robust performance under system perturbations and observation noise.

Abstract

In this work we investigate reinforcement learning (RL) as a framework for the robust control of parametrized dynamical systems in presence of measurements and model uncertainties. High-dimensional state spaces, expensive numerical solvers, the partial knowledge of the governing equations, and the dependence on physical parameters that may be uncertain or difficult to estimate accurately, make the use of standard RL approaches computationally unfeasible. Indeed, lack of robustness and poor generalization across parameter variations are further amplified in presence of noisy or incomplete measurements, ultimately hampering control performance. To address these challenges, we introduce HypEMBER, a novel RL framework based on the combination of hypernetworks and ensemble learning. In the proposed approach, both the policy and value functions are represented through hypernetworks that generate the weights of the underlying models conditioned on the physical parameters of the system, thereby enabling parametric generalization across different dynamical regimes. In addition, an ensemble of policy and value approximators is employed to quantify epistemic uncertainty, leading to improved exploration strategies and enhanced robustness during and after training. The performance of the proposed framework is assessed on two representative parametrized control problems: (i) the one-dimensional Kuramoto-Sivashinsky equation and (ii) a particle-navigation task in a two-dimensional time-dependent gyre flow, focusing on robustness with respect to measurement noise and parameter misspecification. Numerical results demonstrate that HypEMBER consistently improves training stability and sample efficiency, while achieving superior robustness to uncertainties affecting both the system dynamics and the available observations, in comparison with state-of-the-art RL methods.

Problem

• robust control
• parametrized dynamical systems
• reinforcement learning
• model uncertainty
• measurement noise

Innovation

• hypernetworks
• ensemble learning
• robust reinforcement learning
• parametrized dynamical systems
• epistemic uncertainty