StrataCL: Fabric-Native Communication Library for Production Supernodes

2026-07-28

AI Summary

This work addresses communication bottlenecks in distributed AI training, which often stem from the separation between user buffers and communication buffers, leading to redundant data copies or high registration overhead. To overcome this, the authors propose a native Fabric communication library tailored for supernode architectures, featuring an innovative “allocate-and-register” mechanism that enables direct communication from user buffers without data copying. The design further integrates NPU-core load-balanced partitioning with NPU-driven SDMA offloading to co-optimize communication operators holistically. Evaluated on Huawei CloudMatrix384, the approach achieves a 1.6× improvement in collective communication bandwidth and a 1.4× increase in MoE scheduling/merging bandwidth. It also delivers a 1.9× higher LLM inference throughput, a 2.2× reduction in P99 TTFT latency, and 1.4× and 1.3× faster training iteration times for LLMs and recommendation systems, respectively.

Abstract

Modern distributed AI workloads run across hundreds of accelerators, making communication a major bottleneck. Existing communication libraries remain largely buffer-centric because user and communication buffers are managed separately, causing redundant data copies or costly user-buffer registration. This paper presents StrataCL, a zero-redundancy and fabric-native communication library for production supernodes. StrataCL introduces registration-on-allocation to realize user-buffer direct communication, and designs communication operators with workload-balanced NPU-core partitioning and NPU-driven SDMA offloading to exploit supernode architecture features. On the Huawei CloudMatrix384, StrataCL improves collective bus bandwidth by up to 1.6x and improves MoE dispatch/combine bus bandwidth by up to 1.4x. Across three production workloads, StrataCL improves LLM inference throughput by 1.9x, reduces P99 TTFT by 2.2x, and reduces LLM and Recsys training iteration time by 1.4x and 1.3x, respectively.

Problem

• communication bottleneck
• buffer management
• distributed AI
• redundant data copies
• user-buffer registration

Innovation

• StrataCL
• fabric-native
• registration-on-allocation
• NPU-driven SDMA offloading
• zero-redundancy communication