PowerScale: Energy-Efficient Geo-Distributed Model Training with Federated Datacenter Power

2026-07-28

AI Summary

This work addresses the challenge that the energy consumption of large-scale AI training often exceeds the power capacity of a single data center, while existing cross-regional training approaches suffer from inefficient synchronization mechanisms that incur high communication overhead and underutilized resources. To overcome these limitations, the authors propose PowerScale, a novel system featuring a power- and network-aware hierarchical aggregation architecture. PowerScale dynamically adjusts its synchronization strategy based on network proximity and local power availability through regional cluster partitioning, hybrid synchronous-asynchronous aggregation, and adaptive synchronization frequency control. Evaluated in a 100-node simulation, PowerScale achieves up to a 3.9× reduction in energy consumption while maintaining comparable convergence speed.

Abstract

The power demands of large-scale AI training increasingly exceed the capacity of any single data center, making geo-distributed training across power-constrained sites a practical necessity. Prior work optimizes such training mainly for time-to-accuracy using single-tier aggregation, where every site exchanges model updates directly with a central aggregator over the WAN each synchronization round, without accounting for the energy required to reach convergence. Single-tier aggregation is fundamentally energy-inefficient because synchronization barriers force faster sites to idle, full WAN updates dominate communication energy at scale, and fixed synchronization frequency keeps paying the same communication cost even when updates shrink late in training. To address these inefficiencies, we present PowerScale, a hierarchical aggregation system that exploits the latency hierarchy of wide-area networks. PowerScale organizes sites into regional clusters and applies a Sync-Async synchronization modality: sites synchronize frequently with a nearby cluster aggregator over fast local links, while cluster aggregators push pre-aggregated updates asynchronously to a global aggregator over the WAN. PowerScale forms clusters based on both network proximity and power availability, and uses an adaptive synchronization policy that reduces communication energy by adjusting how often clusters synchronize to training progress. This structure shortens synchronization barriers and replaces per-site WAN transmissions with fewer, pre-aggregated transmissions at a lower frequency. We evaluate PowerScale at 100-site scale in a Flower-based simulation environment. PowerScale matches or slightly improves time-to-accuracy compared with single-tier baselines while reducing energy consumption by up to 3.9x.

Problem

• geo-distributed training
• energy efficiency
• federated datacenter power
• model synchronization
• large-scale AI training

Innovation

• hierarchical aggregation
• geo-distributed training
• energy-efficient AI
• adaptive synchronization
• federated datacenter power