CODA: Algorithm-Hardware Co-design for Edge Video Diffusion via NMP-Enabled Compute-Cache Operator Disaggregation

2026-07-16

AI Summary

Deploying video diffusion models on edge devices is challenging due to the compute-intensive iterative Transformer-based denoising and the inefficiency of existing cross-timestep caching strategies, which cause GPU memory overflow and excessive PCIe communication under tight memory constraints. This work proposes a compute-cache operator decoupling architecture that offloads the compute-intensive path to the xPU and the memory-intensive caching path to a near-memory engine adjacent to DIMMs. By exploiting the independence of the classifier-free guidance (CFG) branches, the approach enables parallel execution of computation and caching. Integrated with near-memory processing, cache access coalescing, and cross-timestep cache optimization, the design substantially reduces communication overhead and serial bottlenecks. Experiments demonstrate up to 1.80× end-to-end speedup and 1.74× energy efficiency improvement while preserving generation quality.

Abstract

Deploying Video Diffusion Models (VDMs) on edge devices is appealing for localized and privacy-preserving generation, but their iterative Transformer-based denoising remains too slow for practical local inference. Cross-Timestep Caching (CTC) has emerged as a promising direction for reducing redundant computation, reusing activations across adjacent denoising steps rather than modifying model weights, while largely preserving generation fidelity. However, on memory-constrained edge GPUs, CTC requires a massive cache footprint that quickly exceeds on-device VRAM and forces the cache into host memory. More fundamentally, cache operators remain tightly interleaved and chain-dependent with native compute operators, so naive near-memory offloading still incurs repeated PCIe exchanges for residual and fusion computations, turning cache reuse into a communication- and serialization-bound execution flow. We therefore propose CODA, an algorithm-hardware co-designed architecture centered on Compute-Cache Operator Disaggregation. CODA separates dense compute paths and memory-bound cache paths across the xPU and a lightweight DIMM-side near-memory engine, reorganizes fragmented cache activity into hardware-friendly coalesced segments, and exploits Classifier-Free Guidance (CFG) branch independence to overlap xPU compute with cache-side execution. Experiments show that CODA achieves up to 1.80x end-to-end speedup and 1.74x higher energy efficiency, while preserving competitive generation quality compared with a state-of-the-art caching algorithm.

Problem

• Video Diffusion Models
• Edge Inference
• Cross-Timestep Caching
• Memory Bottleneck
• Operator Interleaving

Innovation

• algorithm-hardware co-design
• compute-cache disaggregation
• near-memory processing
• video diffusion models
• cross-timestep caching