DMG: A Scalable and Efficient Memory-Disaggregated Graph Processing System

2026-07-22

AI Summary

Traditional graph processing systems are constrained by monolithic architectures, where tightly coupled resources lead to low utilization. Existing memory-disaggregated approaches suffer from poor scalability and high cache overhead. This work proposes DMG, the first practical memory-disaggregated graph processing system, which introduces a disaggregation-friendly graph storage layout, an adaptive update coordination mechanism, and a two-level load management strategy to enable efficient graph access, low-overhead update propagation, and dynamic load balancing. DMG is the first system to support elastic scaling across multiple compute and memory nodes while significantly reducing cache requirements without sacrificing performance. Experimental results demonstrate that DMG achieves up to 4.9× higher performance and reduces cache footprint by up to 18.9× compared to the state-of-the-art systems.

Abstract

Traditional graph processing systems are built on monolithic servers, which couple a fixed ratio of compute and memory resources but often result in resource under-utilization in data centers. Although the disaggregated memory (DM) architecture has emerged to address this inefficiency, we identify that existing graph processing systems on DM remain highly impractical. They rely on unscalable architectures that fail to scale beyond a single memory node and a single compute node, and they require compute-side caches that are orders of magnitude larger than conventional practice in DM. To this end, this paper presents DMG, the first practical graph processing system on DM, which demonstrates superior system scalability and cache efficiency while delivering high performance. To improve efficiency of graph retrieval on DM, DMG proposes a DM-friendly graph store with retrieval optimizations. To mitigate costly update propagation, DMG presents an adaptive update coordinator that coordinates compute and memory nodes to perform update propagation with low overhead. To enable fast and effective load balancing, DMG employs a two-stage workload manager that includes a coarse-grained initial partitioning and a fine-grained runtime re-scheduling. Experimental results substantiate that compared with the state-of-the-art DM-based graph processing system, DMG can elastically scale up both compute and memory resources, delivering up to 4.9X better performance and accommodating graphs with ever-increasing sizes; meanwhile, it effectively tames the compute-side cache demands by up to 18.9X, positioning itself as a DM-ready solution in practice.

Problem

• memory-disaggregated
• graph processing
• scalability
• cache efficiency
• resource under-utilization

Innovation

• Memory Disaggregation
• Graph Processing
• Scalable Architecture
• Cache Efficiency
• Load Balancing