Ensemble based Closed-Loop Optimal Control using Physics-Informed Neural Networks

2025-10-20

AI Summary

Solving the Hamilton–Jacobi–Bellman (HJB) equation for optimal control of nonlinear dynamical systems remains challenging due to its analytical intractability and high computational cost in numerical methods. To address this, we propose a multi-stage physics-informed neural network (PINN)-based ensemble learning framework that jointly learns the optimal cost function and the corresponding closed-loop control policy in an end-to-end manner. Departing from conventional stability-inducing regularization terms, our method enables both single-policy and ensemble-policy deployment, significantly enhancing robustness against state disturbances, measurement noise, and initial-condition sensitivity. By tightly integrating prior physical knowledge—encoded via the HJB equation—with data-driven learning, the approach achieves knowledge- and data-cooperative optimal control. Extensive experiments demonstrate high accuracy and strong generalization across infinite-horizon settings, diverse initial conditions, and perturbed dynamics.

Technology Category

• Search and Optimization: Learning to Search
• Intelligent Robots: Learning & Optimization for ROB
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Economics, Online Markets and Human Computation: Social networks and social learning
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The objective of designing a control system is to steer a dynamical system with a control signal, guiding it to exhibit the desired behavior. The Hamilton-Jacobi-Bellman (HJB) partial differential equation offers a framework for optimal control system design. However, numerical solutions to this equation are computationally intensive, and analytical solutions are frequently unavailable. Knowledge-guided machine learning methodologies, such as physics-informed neural networks (PINNs), offer new alternative approaches that can alleviate the difficulties of solving the HJB equation numerically. This work presents a multistage ensemble framework to learn the optimal cost-to-go, and subsequently the corresponding optimal control signal, through the HJB equation. Prior PINN-based approaches rely on a stabilizing the HJB enforcement during training. Our framework does not use stabilizer terms and offers a means of controlling the nonlinear system, via either a singular learned control signal or an ensemble control signal policy. Success is demonstrated in closed-loop control, using both ensemble- and singular-control, of a steady-state time-invariant two-state continuous nonlinear system with an infinite time horizon, accounting of noisy, perturbed system states and varying initial conditions.

Problem

• Solving Hamilton-Jacobi-Bellman equation for optimal control
• Developing ensemble PINN framework without stabilizer terms
• Controlling nonlinear systems with noisy states and perturbations

Innovation

• Ensemble framework learns optimal cost-to-go via HJB
• Physics-informed neural networks without stabilizer terms
• Controls nonlinear systems using ensemble or singular signals