Plasma Shape Control via Zero-shot Generative Reinforcement Learning

2025-10-20

AI Summary

Conventional PID controllers exhibit poor adaptability in plasma shape control, while existing reinforcement learning (RL) methods suffer from weak generalization and require task-specific retraining. Method: This paper proposes a zero-shot generative RL framework leveraging large-scale offline PID demonstration data. It innovatively integrates Generative Adversarial Imitation Learning (GAIL) with Hilbert-space representation learning to embed geometric structure into the latent space, enabling zero-shot cross-trajectory policy transfer without online fine-tuning. Contribution/Results: The framework achieves stable and precise tracking of diverse plasma shape reference trajectories on the HL-3 tokamak simulator. It significantly enhances control flexibility and historical data utilization efficiency, offering a scalable, robust paradigm for intelligent control of fusion devices.

Technology Category

• Intelligent Robots: Behavior Learning & Control
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Traditional PID controllers have limited adaptability for plasma shape control, and task-specific reinforcement learning (RL) methods suffer from limited generalization and the need for repetitive retraining. To overcome these challenges, this paper proposes a novel framework for developing a versatile, zero-shot control policy from a large-scale offline dataset of historical PID-controlled discharges. Our approach synergistically combines Generative Adversarial Imitation Learning (GAIL) with Hilbert space representation learning to achieve dual objectives: mimicking the stable operational style of the PID data and constructing a geometrically structured latent space for efficient, goal-directed control. The resulting foundation policy can be deployed for diverse trajectory tracking tasks in a zero-shot manner without any task-specific fine-tuning. Evaluations on the HL-3 tokamak simulator demonstrate that the policy excels at precisely and stably tracking reference trajectories for key shape parameters across a range of plasma scenarios. This work presents a viable pathway toward developing highly flexible and data-efficient intelligent control systems for future fusion reactors.

Problem

• Developing versatile plasma shape control without retraining
• Overcoming limited generalization in traditional control methods
• Creating zero-shot policy for diverse trajectory tracking tasks

Innovation

• Zero-shot generative reinforcement learning for plasma control
• Combining GAIL with Hilbert space representation learning
• Foundation policy enables diverse tracking without fine-tuning