DGNA: Dissecting GPU NUMA Architecture through Microbenchmarking and Data Analysis

2026-07-22

AI Summary

This work addresses the limitations in GPU memory efficiency stemming from a lack of transparent understanding of NUMA architectures within the L2 cache and DRAM hierarchy, which hinders both application-level optimizations and architectural design. To overcome this, the paper proposes a vendor-agnostic framework that combines microbenchmarks with Gaussian mixture models to accurately measure and deconvolve memory latency distributions. Applying this methodology to NVIDIA A100 and H100 GPUs, the study provides the first systematic characterization of internal NUMA node organization, the mapping between streaming multiprocessors (SMs) and NUMA domains, and the underlying cache coherence mechanisms. These insights offer critical foundations for high-performance GPU code optimization and the development of accurate architectural simulators.

Abstract

Graphics Processing Units (GPUs), due to their immense parallel processing capabilities, have become essential across various fields, including gaming and artificial intelligence. With significant advancements in GPU cores, GPU memory efficiency has lagged, resulting in bottlenecks that can limit workload efficiency. To bridge this gap, a deep understanding of GPU memory architectures, particularly Non-Uniform Memory Access (NUMA) mechanisms within L2 and DRAM, is essential for optimizing applications, designing new architectures, and building accurate simulators. However, the latest GPU hardware from vendors like NVIDIA and AMD is still a black-box, making it challenging for researchers to understand the details of their design. In this paper, we introduce DGNA, a methodology designed to unveil the NUMA architecture of the GPU memory hierarchy through microbenchmarking and data analysis. Specifically, we propose an approach to measuring the latency of L2 caches and DRAM without relying on the intrinsic instructions of the architecture and apply a Gaussian mixture model to filter out outliers and accurately determine latency distributions. We apply DGNA on NVIDIA's A100 and H100 GPUs, revealing NUMA node architecture, SM-NUMA relationships, and NUMA-aware memory allocation strategies used to maintain cache coherence. To the best of our knowledge, this is the first paper to detail the NUMA architecture within the GPU memory subsystem.

Problem

• GPU
• NUMA
• memory architecture
• black-box
• latency

Innovation

• GPU NUMA
• microbenchmarking
• latency measurement
• Gaussian mixture model
• memory hierarchy