Graph-Enhanced Model-Free Reinforcement Learning Agents for Efficient Power Grid Topological Control

2025-03-26

AI Summary

Modern power systems face challenges in source-grid-load coordinated topology control, characterized by high-dimensional discrete action spaces, stringent operational constraints, and the absence of prior analytical models. Method: This paper proposes a graph-enhanced model-free deep reinforcement learning framework. It introduces a masked topology action space, integrates state-logic guidance with an adversarial multi-agent training paradigm, and employs graph neural networks (GNNs) to encode grid topology. Combined with PPO/SAC variants and dynamic observation formalization, the approach enables end-to-end autonomous topology regulation. Contribution/Results: Evaluated across 20 five-substation simulation scenarios, the method achieves an average 12.7% reduction in network losses, zero instability events, and consistently outperforms baseline methods. These results validate the feasibility and engineering applicability of model-free autonomous decision-making for real-time power grid control.

Technology Category

Multiagent Systems: Adversarial AgentsPlanning, Routing, and Scheduling: Planning/Scheduling and LearningSearch and Optimization: Learning to Search

Application Category

Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphsEconomics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystemsSystems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments

Abstract

The increasing complexity of power grid management, driven by the emergence of prosumers and the demand for cleaner energy solutions, has needed innovative approaches to ensure stability and efficiency. This paper presents a novel approach within the model-free framework of reinforcement learning, aimed at optimizing power network operations without prior expert knowledge. We introduce a masked topological action space, enabling agents to explore diverse strategies for cost reduction while maintaining reliable service using the state logic as a guide for choosing proper actions. Through extensive experimentation across 20 different scenarios in a simulated 5-substation environment, we demonstrate that our approach achieves a consistent reduction in power losses, while ensuring grid stability against potential blackouts. The results underscore the effectiveness of combining dynamic observation formalization with opponent-based training, showing a viable way for autonomous management solutions in modern energy systems or even for building a foundational model for this field.

Problem

• Optimizing power grid operations without expert knowledge
• Reducing power losses while maintaining grid stability
• Developing autonomous management for modern energy systems

Innovation

• Model-free reinforcement learning for grid control
• Masked topological action space for cost reduction
• Dynamic observation with opponent-based training