Beyond HBM-on-GPU: Thermal Design Envelope for 3D Volumetric DRAM-on-GPU Integration

2026-09-21

AI Summary

本文解决了GPU在AI和HPC工作负载中因容量、带宽及热限制导致的扩展问题，通过3D体积DRAM-on-GPU集成设计及其热管理方法来优化性能。

Abstract

The scaling of GPUs for AI and HPC workloads is increasingly constrained by the capacity, bandwidth, and thermal limits of both 2.5D HBM-GPU and direct-stacked 3D HBM-on-GPU integration. This work establishes the thermal design envelope for 3D volumetric DRAM-on-GPU integration, in which vertically oriented DRAM dies and interleaved cooling cavities reshape heat flow and memory interfacing above the GPU. Using a package-level thermal model anchored to a consistent HBM-on-GPU baseline and driven by a realistic reticle-scale non-uniform GPU power map, we quantify the key parameters governing thermal feasibility. Stack height is the dominant limiter of peak temperature, while cooling-cavity conductivity shifts the feasible region, and mold insertion and stack orientation further modulate thermal behavior. A distributed memory-controller and network-on-chip tier introduces only a moderate thermal penalty. Although die-level parallelism increases bandwidth, the reduction in simulated training time saturates once execution becomes compute-bound. These results define a bounded co-design space across bandwidth, capacity, and thermal constraints for 3D volumetric DRAM-on-GPU integration.

Problem

• Thermal Design Envelope
• 3D Volumetric DRAM-on-GPU
• Heat Flow
• Memory Interfacing
• Thermal Limits

Innovation

• 3D Volumetric DRAM-on-GPU
• Thermal Design Envelope
• Cooling Cavities
• Package-level Thermal Model
• Memory-Controller and Network-on-Chip