PowerMarketJax: A JAX Benchmark Suite for Multi-Agent Reinforcement Learning in Power Markets

2026-09-29

AI Summary

This study addresses critical bottlenecks in existing multi-agent reinforcement learning (MARL) environments for electricity markets, including limited scenario diversity, oversimplified clearing mechanisms, and low CPU-based computational efficiency. To overcome these limitations, this work proposes the first end-to-end integration of market simulation and policy training within the JAX framework. We construct a comprehensive benchmark suite encompassing five types of electricity markets, with unified implementations of clearing and pricing rules that support GPU-accelerated massively parallel training. The proposed approach achieves a high parallelism configuration of 1024×1200, delivering a 33-fold speedup over CPU baselines. This substantial improvement in large-scale simulation efficiency effectively reveals the critical influence of market design on bidding behaviors.

Abstract

Power markets are a natural testbed for multi-agent reinforcement learning (MARL), where multiple self-interested participants repeatedly submit bids. A market-clearing mechanism then determines dispatch and prices subject to power grid constraints and market settlement rules. However, existing MARL environments typically focus on a single market setting, implement simplified clearing mechanisms, or rely on CPU-based optimization solvers that slow large-scale training and limit the systematic study of bidding strategies and market behavior. We introduce PowerMarketJax, a benchmark suite for MARL across five power markets: day-ahead wholesale, real-time balancing, ancillary services, peer-to-peer double auctions, and local flexibility. Each environment implements its own clearing, pricing, and settlement rules while providing a common framework for learning and evaluation. We find that learned bidding behavior depends strongly on the market design: independent learners can miss better strategies when gains require many agents to change together, when more profitable strategies lie beyond a region of lower profit, or when profits disappear as more agents adopt the same strategy. PowerMarketJax implements both market simulation and policy training in JAX, allowing the entire pipeline to run on the GPU with 1,024 X 1,200 parallelisms across both environments and market participants, achieving up to 33X speedup over CPU-based baselines. Our open-source benchmark is available at: https://github.com/powermarketjax/PowerMarketJax.

Problem

• Multi-Agent Reinforcement Learning
• Power Markets
• Benchmark Suite
• Market Clearing
• GPU Acceleration

Innovation

• Multi-Agent Reinforcement Learning
• JAX
• Power Markets
• GPU Acceleration
• Benchmark Suite