Task-optimal data-driven surrogate models for eNMPC via differentiable simulation and optimization

2024-03-21

arXiv.org

AI Summary

Real-time economic nonlinear model predictive control (eNMPC) faces significant computational burdens from first-principles dynamic models, hindering simultaneous satisfaction of real-time implementation requirements and strict constraint enforcement. Method: This paper proposes a task-driven Koopman surrogate modeling framework that jointly embeds differentiable simulation and eNMPC optimization into surrogate training. Leveraging implicit gradient propagation, it enables end-to-end structured learning—bypassing conventional reinforcement learning paradigms while inherently guaranteeing hard constraint satisfaction. Contribution/Results: Theoretical integration combines Koopman operator theory with gradient-based policy optimization. Experimental evaluation on the CSTR benchmark demonstrates that the proposed method achieves comparable economic performance to state-of-the-art approaches while eliminating all constraint violations, thereby substantially improving controller reliability and real-time feasibility.

Technology Category

Search and Optimization: Sampling/Simulation-based SearchConstraint Satisfaction and Optimization: Constraint Learning and AcquisitionGame Theory and Economic Paradigms: Cooperative Game Theory

Application Category

Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applicationsUser Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systemsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Mechanistic dynamic process models may be too computationally expensive to be usable as part of a real-time capable predictive controller. We present a method for end-to-end learning of Koopman surrogate models for optimal performance in a specific control task. In contrast to previous contributions that employ standard reinforcement learning (RL) algorithms, we use a training algorithm that exploits the differentiability of environments based on mechanistic simulation models to aid the policy optimization. We evaluate the performance of our method by comparing it to that of other training algorithms on an existing economic nonlinear model predictive control (eNMPC) case study of a continuous stirred-tank reactor (CSTR) model. Compared to the benchmark methods, our method produces similar economic performance while eliminating constraint violations. Thus, for this case study, our method outperforms the others and offers a promising path toward more performant controllers that employ dynamic surrogate models.

Problem

• Develop task-optimal surrogate models for eNMPC
• Use differentiable simulation for policy optimization
• Achieve economic performance without constraint violations

Innovation

• End-to-end learning of Koopman surrogate models
• Differentiable simulation for policy optimization
• Constraint-free economic performance in eNMPC