ComFuse: Fusing Complex Memory-Intensive Subgraphs with Compute-Intensive Kernels For Modern GPU Architectures

2026-08-04

AI Summary

This work addresses the limitations of existing deep learning compilers, which optimize compute-intensive and memory-intensive operators in isolation, leading to rigid fusion boundaries that hinder cross-operator optimization and on-chip data reuse. To overcome this, the paper proposes a novel operator fusion strategy that, for the first time, enables synergistic fusion of both types of subgraphs. The approach automatically integrates complex, dependency-rich memory-intensive subgraphs with compute-intensive operators—such as back-to-back GEMMs—into high-performance GPU kernels. Built upon automated GPU compilation techniques, the method lowers high-level tensor programs into optimized fused kernels and employs concurrent scheduling to hide memory latency. Experimental results demonstrate that the generated kernels outperform those produced by TorchInductor on complex workloads like post-norm transformers and support a broader range of fusion patterns.

Abstract

Modern deep learning workloads increasingly comprise heterogeneous computation graphs that combine compute-intensive operators with memory-intensive subgraphs. Existing deep learning compilers typically optimize these operator classes separately, creating rigid fusion boundaries that limit cross-operator optimization and on-chip data reuse. We observe that downstream memory-intensive operations can execute concurrently with compute-intensive operators, allowing their execution to be hidden behind computation; however, automatically exploiting this opportunity poses new compilation challenges. In this paper, we present ComFuse, an automated GPU compilation system that employs a novel operator fusion strategy to generate high-performance kernels for complex graph structures comprising compute-intensive operators and dependency-rich, memory-intensive elementwise-reduction subgraphs. ComFuse further supports the fusion of back-to-back GEMM (B2BGEMM) patterns, extending its applicability to more complex compute-memory interaction patterns. Additionally, it automatically lowers high-level tensor subprograms into optimized fused kernels, reducing the need for manual kernel engineering. Experimental results show that the fused kernels generated by ComFuse outperform those produced by TorchInductor across post-norm workloads and various complex computation scenarios, while supporting more flexible fusion patterns.

Problem

• operator fusion
• compute-intensive operators
• memory-intensive subgraphs
• GPU compilation
• heterogeneous computation graphs

Innovation

• operator fusion
• compute-memory co-optimization
• GPU compilation
• memory-intensive subgraphs
• B2BGEMM