C2P-Cache: Scalable GPU L1 Cache Sharing via Concurrent Candidate Pruning

2026-07-31

AI Summary

This work addresses the inefficiency of traditional GPU private L1 caches, which fail to exploit data reuse across streaming multiprocessors (SMs), leading to redundant L2 cache accesses. To overcome this limitation, the authors propose a scalable L1 sharing mechanism that decomposes chip-wide remote hit detection into two efficient phases—filtering and verification—via lightweight candidate pruning. The core innovations include a concurrent pruning strategy based on Bloom filter snapshots to avoid costly global exact searches, coupled with bit-sliced matching, tiled replication of snapshot matrices, and selective probing to enable highly parallel handling of concurrent miss requests. Experimental results demonstrate that the proposed approach improves instructions per cycle (IPC) by 23.5% on average across diverse GPU workloads, with gains reaching up to 49.7%, significantly enhancing performance for L2 latency-sensitive applications.

Abstract

Modern GPUs rely on private per-SM L1 caches and a shared L2 cache, but this organization obscures cross-SM reuse: an L1 miss is typically forwarded to L2 even when the requested line already resides in a peer L1 cache, leading to redundant L2 access. Prior GPU L1-sharing designs attempt to recover such reuse through exact or broad remote-hit searches, which become increasingly difficult to scale and can interfere with the critical L1 miss path under high concurrency. %miss handling as more caches participate and more misses arrive concurrently. We observe that eliminating redundant L2 accesses does not require exact, chip-wide knowledge of private L1 contents. Instead, it requires only sufficient visibility to sharply narrow down a small set of candidate caches, leaving exact confirmation to a much smaller number of L1s. Based on this insight, we propose C2P-Cache, a scalable GPU L1-sharing mechanism that transforms remote-hit discovery from a chip-wide exact search problem into a lightweight filtering-and-confirmation process. C2P-Cache maintains compact Bloom-filter-based snapshots of private L1 tags, performs parallel chip-wide candidate filtering, and selectively probes only a small number of likely peer caches. To sustain high concurrency, C2P-Cache organizes filtering as bit-sliced matching over a banked and replicated snapshot matrix, enabling efficient, parallel processing of many concurrent misses without interfering with normal L1 accesses. Across a wide range of GPU workloads, C2P-Cache improves instructions per cycle (IPC) by up to 49.7\% and by 23.5\% on average for applications with high remote-L1 reuse and strong sensitivity to L2 latency, demonstrating that lightweight, scalable filtering can effectively unlock cross-SM reuse with modest overhead.

Problem

• GPU caching
• L1 cache sharing
• cross-SM reuse
• redundant L2 access
• scalability

Innovation

• GPU cache sharing
• Bloom filter
• concurrent filtering
• L1 cache coherence
• scalable architecture