memory management

BVH performance is highly sensitive to data layout, yet existing systems tightly couple layout design with traversal logic, constraining layout optimization within algorithm-specific implementations and hindering simultaneous performance gains and portability. This paper introduces Scion—a domain-specific language and compiler that decouples BVH layout specification from traversal algorithms, enabling architecture-agnostic layout declarations and automatic optimization. Its core contribution is the first full abstraction of BVH layout coupled with cross-platform joint optimization, uncovering a novel layout that achieves Pareto optimality across diverse ray-tracing workloads. Experiments demonstrate that the optimal layout varies dynamically with traversal algorithm, hardware architecture, and workload characteristics. Scion-generated layouts consistently deliver Pareto-superior trade-offs between performance and memory footprint across mainstream CPUs and GPUs compared to conventional hand-tuned layouts.

To address the challenge of optimizing data placement in HBM+DDR heterogeneous memory systems, this paper proposes a lightweight, non-intrusive, application-level memory analysis and control framework. Leveraging a detailed memory subsystem model on Intel Sapphire Rapids platforms—integrated with empirical bandwidth/latency measurements, runtime allocation tracing, and policy injection—the work quantifies, for the first time, the performance inflection point of HBM-DRAM co-utilization: retaining only 60–75% of critical data in HBM achieves 90% of the platform’s peak performance. The approach requires no source-code modification or recompilation. Evaluated across multiple benchmarks, it significantly improves performance for memory-intensive applications while reducing HBM resource consumption by over 30%. This establishes a deployable, fine-grained data placement optimization paradigm for heterogeneous memory systems.

Data transfer between the GPU and host memory is significantly slower than computational speed, becoming a major performance bottleneck for SPH solvers. To address this, this work proposes a host-side particle memory layout optimization tailored for GPU offloading. By analyzing GPU kernel access patterns and particle attribute types, the conventional Array-of-Structures (AoS) layout is decomposed into multiple fine-grained sub-structures (Split AoS), combined with a data compression strategy to substantially reduce the overhead of data reorganization before and after transfers. Experimental results demonstrate that the proposed approach reduces data packing time by 20%–40% and decreases overall GPU offloading latency by 12%–25%, thereby significantly enhancing heterogeneous computing efficiency.

To address performance, energy-efficiency, and latency bottlenecks arising from data movement between main memory and processors in data-intensive computing, this paper proposes and systematically advances the practical deployment of Processing-in-Memory (PIM). We introduce a unified PIM architecture framework that innovatively integrates 3D-stacked logic layers, on-die in-memory computing units, memory controller–integrated accelerators, on-chip ECC-enhanced DRAM, and hardware-level RowHammer mitigation. This holistic design significantly reduces data movement overhead, achieving energy-efficiency improvements of several-fold to over an order of magnitude across representative workloads, while ensuring scalability and reliability. The framework establishes a foundation for low-power, high-throughput memory-compute infrastructure applicable to both server and mobile platforms. By bridging architectural innovation with system-level implementation, our work accelerates the transition of PIM from theoretical concept to real-world deployment.

The performance characteristics and architectural behaviors of cache-coherent interconnects—particularly Compute Express Link (CXL)—remain poorly understood in multi-vendor heterogeneous systems (e.g., CPU + CXL memory devices). Method: We construct a cross-vendor heterogeneous server cluster and propose Heimdall, the first fine-grained memory performance analysis framework tailored for CXL systems, accompanied by a lightweight microbenchmark suite. Through empirical measurement of CXL 3.0 protocol stack–hardware co-behavior, we systematically characterize memory latency, bandwidth, and coherence semantics across mainstream CXL devices. Contribution/Results: We uncover three previously unknown architectural blind spots and implicit protocol stack constraints. Leveraging these insights, we devise practical, workload-aware memory scheduling strategies for database and AI inference workloads. Our work provides both theoretical foundations and actionable guidelines for designing and optimizing cache-coherent heterogeneous systems.

This work addresses the significant degradation in inference throughput caused by GPU memory constraints when concurrently deploying multiple large language models on shared heterogeneous hardware, where resource scheduling, model offloading, and preemption become critical bottlenecks. Through empirical methodologies—including cross-platform performance profiling, layer-wise offloading experiments, and fine-grained decomposition of preemption overhead—the study systematically uncovers, for the first time, the nonlinear relationship between offloading and throughput decline. It further identifies model state reloading as the primary source of preemption overhead. The findings reveal that smaller models are more sensitive to reduced GPU residency, and that such overhead is jointly influenced by model architecture and hardware characteristics. These insights motivate a scheduler design that integrates model-specific sensitivity with data migration costs, offering crucial guidance for building efficient multi-model serving systems.

The PIM community has long assumed interchangeability between bit-parallel (BP) and bit-serial (BS) data layouts, lacking workload-aware, systematic criteria for layout selection. Method: This work introduces the first workload-characteristic-driven data layout selection methodology, employing area-equivalent, cycle-accurate architectural modeling and comprehensive evaluation via MIMDRAM microbenchmarks and PIMBench application workloads. Contribution/Results: We identify the performance boundary between BP and BS layouts: BP excels in control-flow-intensive workloads, whereas BS significantly outperforms BP in low-precision AI computations. Our findings refute the implicit assumption of a universally optimal layout, establishing a theoretical foundation and practical framework for hybrid layout design. Crucially, we demonstrate that no single layout is optimal across diverse workloads—layout selection must be workload-specific.

This paper systematically investigates the performance trade-offs between Processing-in-Memory (PIM) and CXL-based PIM (CXL-PIM) architectures. It addresses the fundamental tension: conventional PIM incurs high explicit data movement overhead, whereas CXL-PIM offers a unified address space but suffers from elevated memory access latency. To resolve this, the authors propose an end-to-end evaluation methodology that integrates empirical measurements from real PIM hardware with trace-driven CXL channel modeling, enabling large-scale benchmarking across mainstream workloads. Their analysis reveals, for the first time, that the amortization effect of unified addressing on interconnect latency is dynamic—varying with dataset size and execution phase—and can cause performance inversion between the two architectures. Building on this, they quantitatively characterize the boundary conditions defining the performance crossover points. The findings provide principled, quantifiable guidance for near-memory system design and uncover a novel architectural design space.

This work addresses the performance limitations of data-intensive applications in edge computing, which often suffer from poor memory locality or excessive memory footprint. To overcome these challenges, the authors propose a hardware-software co-design that integrates a tensor memory engine into the general-purpose CPU datapath. This engine enables runtime dynamic reconfiguration of memory layouts to significantly enhance cache locality, without requiring application code modifications, offloading computation to memory, or incurring memory overhead. By preserving the CPU’s role as the primary compute unit, the approach effectively improves overall system performance. Notably, this study presents the first practical implementation of runtime memory layout reconfiguration on commercial SoC/FPGA platforms, demonstrating both innovation and real-world applicability.

To address the latency overhead and performance bottlenecks imposed by heterogeneous memory controllers in NVRAM-based main memory systems—particularly for big-data and AI workloads—this paper proposes a two-level collaborative prefetching architecture spanning hybrid memory cube (HMC) and on-chip L1 cache. We design a novel multi-level prefetching engine that enables dynamic, synergistic optimization between HMC- and L1-level prefetchers on out-of-order execution processors. Experimental evaluation shows that HMC-only prefetching improves performance by 9%, while the joint HMC+L1 scheme achieves a 12% gain. Prefetch coverage reaches up to 92%, with accuracy improved to 80%. Crucially, this work is the first to reveal the pivotal role of the L1 prefetcher in significantly enhancing off-chip prefetch coverage—a key insight enabling low-latency, high-efficiency prefetching across deep memory hierarchies. Our approach establishes a new paradigm for coordinated, hierarchy-aware prefetching in emerging NVRAM-centric systems.