Hardware-Aware Features for CUTLASS Kernel Selection

2026-09-28

AI Summary

This study addresses the prohibitive cost of auto-tuning and the difficulty of pre-execution selection arising from the vast CUTLASS kernel space. To tackle this, we propose a hardware-aware representation method that incorporates hardware behavior estimation as an inductive bias into the model. By integrating static hardware modeling, gradient boosting, and neural learning-to-rank techniques, the approach enables efficient ranking of kernel candidates. Experimental results demonstrate that, compared to conventional structured baselines and heuristic methods, our solution significantly reduces selection regret by 40% to 64.2%. Furthermore, it exhibits strong transferability across mixed-precision computation and operator fusion scenarios.

Abstract

GPU libraries such as CUTLASS expose tens of thousands of semantically equivalent kernels for a single operation, making exhaustive autotuning expensive and execution-free selection difficult. Existing analytical selectors require hand-designed performance rules, while learned selectors operate on raw configuration parameters and must infer hardware consequences from data. We introduce a hardware-aware representation for CUTLASS kernel selection that augments candidate configurations with statically computable estimates of induced hardware behavior. We construct a dataset of 4.9 million CUTLASS kernels and train gradient-boosted and neural learning-to-rank models to rank candidates within each problem. On held-out exhaustive evaluation problems, hardware-aware representations reduce selection regret by up to 40\% relative to structural baselines and 64.2\% relative to NVIDIA's matrix-multiply heuristics. We further evaluate data-efficient cross-precision and epilogue-fusion transfer within CUTLASS GEMM, showing that explicitly representing candidate-induced hardware behavior provides a useful inductive bias for learned kernel selection.

Problem

• Kernel Selection
• CUTLASS
• GPU
• Autotuning
• Hardware-Aware

Innovation

• Hardware-Aware Representation
• Kernel Selection
• Learning-to-Rank
• CUTLASS
• Inductive Bias