A Hierarchical MARL-Based Approach for Coordinated Retail P2P Trading and Wholesale Market Participation of DERs

2026-04-22

AI Summary

This study addresses the challenge of enabling distributed energy resources (DERs) to simultaneously participate in both retail peer-to-peer (P2P) trading and wholesale markets. To this end, the authors propose a coordinated optimization framework that integrates hierarchical multi-agent deep reinforcement learning (MARL) with Stackelberg game theory. The approach empowers prosumers to autonomously engage in retail P2P auctions while also allowing their aggregation into collective entities for wholesale market participation, thereby preserving individual decision-making autonomy while achieving system-level optimality. By innovatively combining hierarchical MARL with Stackelberg博弈, the framework enhances overall market efficiency and grid operational flexibility, and establishes a resource-efficient, readily deployable intelligent demand response mechanism.

Technology Category

Multiagent Systems: Mechanism DesignSearch and Optimization: Distributed SearchGame Theory and Economic Paradigms: Coordination and Collaboration

Application Category

Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystemsSystems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environmentsResponsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The ongoing shift towards decentralization of the electric energy sector, driven by the growing electrification across end-use sectors, and widespread adoption of distributed energy resources (DERs), necessitates their active participation in the electricity markets to support grid operations. Furthermore, with bi-directional energy and communication flows becoming standard, intelligent, easy-to-deploy, resource-conservative demand-side participation is expected to play a critical role in securing power grid operational flexibility and market efficiency. This work proposes a market engagement framework that leverages a hierarchical multi-agent deep reinforcement learning (MARL) approach to enable individual prosumers to participate in peer-to-peer retail auctions and further aggregate these intelligent prosumers to facilitate effective DER participation in wholesale markets. Ultimately, a Stackelberg game is proposed to coordinate this hierarchical MARL-based DER market participation framework toward enhanced market performance.

Problem

• distributed energy resources
• peer-to-peer trading
• wholesale market participation
• market coordination
• electricity markets

Innovation

• Hierarchical MARL
• Peer-to-Peer Trading
• Distributed Energy Resources
• Stackelberg Game
• Wholesale Market Participation