🤖 AI Summary
This work addresses the lack of unified coordination over the full lifecycle of DMA buffers in existing AI data transfer libraries, which undermines safety and performance under high load. To resolve this, we propose dmaplane, the first system that introduces buffer orchestration as a standalone abstraction within the Linux kernel. It provides a unified /dev/dmaplane user-space API to manage allocation, cross-device sharing, secure deallocation, and synchronization. dmaplane integrates NUMA-aware allocation, a kernel-level RDMA engine, direct GPU BAR mapping, and credit-based flow control, substantially enhancing reliability and throughput. Experiments demonstrate that GPU BAR mapping outperforms cudaMemcpy, RDMA WRITE WITH IMMEDIATE enables efficient cross-machine key-value cache transfers, and the system maintains strong safety guarantees with low overhead even under high load.
📝 Abstract
AI transport libraries move bytes efficiently, but they commonly assume that buffers are already correctly allocated, placed, shared, registered, and safe under completion and teardown pressure. This paper presents dmaplane, a Linux kernel module that makes this missing layer explicit as buffer orchestration. dmaplane exposes a stable kernel UAPI via /dev/dmaplane and composes ring-based command channels, DMA buffer lifecycle management, dma-buf export for cross-device sharing, a kernel-space RDMA engine, NUMA-aware allocation and verification, credit-based flow control, low-overhead observability, and GPU memory integration via PCIe BAR pinning. We evaluate orchestration sensitivity with measurements of NUMA cross-node penalties at DRAM scale, completion-safe flow control under sustained RDMA load, and GPU BAR mapping tiers versus cudaMemcpy. We also demonstrate end-to-end disaggregated inference by transferring KV-cache chunks between two machines using RDMA WRITE WITH IMMEDIATE and reconstructing tensor views on the receiver. RDMA measurements use Soft-RoCE; we distinguish measured results from provider-independent properties by construction.