Defeat the Heap: Zero-Copy Data Movement in AXI4MLIR

2026-06-09

AI Summary

This work addresses the significant runtime overhead in AXI4MLIR caused by redundant non-zero-copy operations due to heap-allocated buffers during data transfers between host and accelerator. To eliminate the intermediate copy stage in host-to-accelerator communication, this study introduces zero-copy support into MLIR’s `accel` dialect for the first time by directly allocating buffers in DMA-mapped memory. The authors reformulate the buffer allocation strategy and extend the MLIR compilation framework to enable DMA-aware buffer allocation and corresponding low-level code generation. Experimental evaluation on a configurable matrix multiplication accelerator demonstrates a two-fold reduction in main memory data movement, substantially improving accelerator utilization.

Abstract

As custom hardware accelerators become increasingly central to machine learning workloads, efficient data transfer is critical for maximizing accelerator performance on linear algebra kernels. AXI4MLIR, an extension of the Multi-Level Intermediate Representation (MLIR) compiler framework for automated generation of host-accelerator driver code, incurs significant runtime overhead due to non-zero-copy CPU-accelerator data movement. During transfers from the host to the accelerator, data is copied from heap-allocated memory buffers into contiguous Direct Memory Access (DMA)-mapped buffers. This work identifies this copy as a redundant staging operation and eliminates it through zero-copy data movement. The optimization extends accel, an MLIR dialect introduced by AXI4MLIR, and implements lowering support that allocates buffers directly within DMA-mapped memory, thereby omitting the staging copy. We evaluate the proposed scheme using a configurable matrix-matrix multiplication accelerator and show that the zero-copy optimization reduces main memory data movement by up to 2x, increasing overall accelerator utilization.

Problem

• zero-copy
• data movement
• AXI4MLIR
• DMA
• heap allocation

Innovation

• zero-copy
• AXI4MLIR
• DMA-mapped memory
• MLIR dialect
• data movement optimization