Automated co-design of high-performance thermodynamic cycles via graph-based hierarchical reinforcement learning

2026-04-13

AI Summary

This work addresses the inefficiency and limited scalability of traditional thermodynamic cycle design, which often relies on expert intuition or exhaustive search. To overcome these limitations, the authors propose an end-to-end automated co-design framework that represents cycles as grammar-constrained graph structures and integrates graph neural networks, physics-informed surrogate models, and hierarchical reinforcement learning with a Manager-Worker architecture to jointly optimize both topology and parameters. This approach uniquely unifies graph-based representation, physical surrogates, and reinforcement learning, offering both high efficiency and broad applicability. Validated on heat pump and heat engine benchmarks, the method not only reproduces established cycles but also discovers 18 and 21 novel configurations, respectively, achieving performance improvements of 4.6% and 133.3% over baseline designs.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Thermodynamic cycles are pivotal in determining the efficacy of energy conversion systems. Traditional design methodologies, which rely on expert knowledge or exhaustive enumeration, are inefficient and lack scalability, thereby constraining the discovery of high-performance cycles. In this study, we introduce a graph-based hierarchical reinforcement learning approach for the co-design of structure parameters in thermodynamic cycles. These cycles are encoded as graphs, with components and connections depicted as nodes and edges, adhering to grammatical constraints. A deep learning-based thermophysical surrogate facilitates stable graph decoding and the simultaneous resolution of global parameters. Building on this foundation, we develop a hierarchical reinforcement learning framework wherein a high-level manager explores structural evolution and proposes candidate configurations, whereas a low-level worker optimizes parameters and provides performance rewards to steer the search towards high-performance regions. By integrating graph representation, thermophysical surrogate, and manager-worker learning, this method establishes a fully automated pipeline for encoding, decoding, and co-optimization. Using heat pump and heat engine cycles as case studies, the results demonstrate that the proposed method not only replicates classical cycle configurations but also identifies 18 and 21 novel heat pump and heat engine cycles, respectively. Relative to classical cycles, the novel configurations exhibit performance improvements of 4.6% and 133.3%, respectively, surpassing the traditional designs. This method effectively balances efficiency with broad applicability, providing a practical and scalable intelligent alternative to expert-driven thermodynamic cycle design.

Problem

• thermodynamic cycles
• design automation
• high-performance cycles
• scalability
• expert-driven design

Innovation

• graph-based representation
• hierarchical reinforcement learning
• thermodynamic cycle co-design
• thermophysical surrogate model
• automated design