roofline modeling

This work addresses the limitations of traditional speed-of-light performance analysis, which relies on error-prone manual derivations and struggles to keep pace with rapid model iteration. The authors propose SOLAR, a novel framework that, for the first time, enables end-to-end automated generation and verification of speed-of-light performance bounds for deep learning models. SOLAR integrates large language models with a deterministic analytical pipeline: an LLM frontend generates Affine Loop IR, constructs einsum graphs, and performs analytical modeling, enabling operator-level coverage, cache-aware analysis, and inverse roofline hardware planning. Experiments on KernelBench, JAX/Flax models, and robotics workloads demonstrate that SOLAR achieves zero speed-of-light violations and effectively guides the identification of optimization opportunities and cross-platform performance exploration.

Large language models (LLMs) struggle to autonomously optimize GPU kernel code without performance context, while conventional profiling tools identify bottlenecks but fail to generate executable optimization strategies. Method: This paper introduces the first GPU kernel auto-optimization framework integrating LLMs with dynamic hardware performance insights—including hardware counter metrics and Roofline model analysis—via structured prompting that precisely encodes bottleneck characteristics to guide high-fidelity, executable code generation. Contribution/Results: Evaluated across 1,640 experiments, the framework achieves performance improvements in over 98.5% of cases, with average speedups ranging from 19.34% to 52.3%; generated code exhibits near-perfect functional correctness. This work bridges the longstanding gap between low-level performance analysis and high-level optimization decision-making, establishing a reproducible, scalable paradigm for AI-driven, system-level GPU code optimization.

This work addresses the limitation of existing GPU kernel optimization benchmarks, which focus solely on software baseline speedups and fail to assess proximity to hardware theoretical limits. Targeting the NVIDIA Blackwell architecture, the authors construct a benchmark suite comprising 235 real-world AI model CUDA kernels and reframe the optimization objective from surpassing software baselines to approaching the hardware Speed-of-Light (SOL) ceiling. Leveraging the SOLAR framework, they derive theoretical SOL bounds for each kernel and introduce the SOL Score to quantify hardware efficiency. To ensure rigor, they design specialized kernels and a sandboxed evaluation environment supporting new data types (BF16, FP8, NVFP4), featuring clock locking, L2 cache flushing, subprocess isolation, and anti-cheating static checks. This effort establishes the first hardware-limit-oriented, cheat-resistant, and reproducible benchmark for multimodal AI kernels, providing a reliable foundation for agent-driven automated optimization.

Accurately identifying computational, memory bandwidth, and memory latency bottlenecks—and quantifying associated resource slack—is critical yet challenging for HPC application performance tuning. This paper introduces the first model-agnostic, instruction-level precise noise-injection framework for bottleneck analysis. Leveraging the LLVM toolchain, it selectively injects computational or memory-access noise instructions to decouple the impact of each resource constraint, enabling fine-grained bottleneck classification and quantitative slack measurement. Unlike prior approaches, it requires no hardware modeling assumptions and is portable across diverse architectures. Evaluated on heterogeneous memory systems—including HBM and DDR—it demonstrates robust effectiveness. The method significantly improves the precision of optimization decisions and hardware selection guidance, addressing key limitations of existing tools in slack quantification and root-cause attribution of performance bottlenecks.

This work addresses the challenge of prolonged and error-prone manual kernel development for emerging AI accelerators, which stems from their use of specialized instruction set architectures (ISAs) and hinders cross-platform portability. To overcome this, the paper introduces the first agent-driven benchmark for kernel generation tailored to novel hardware, featuring a large language model (LLM)-based feedback optimization framework. This framework leverages function calling and iterative refinement to automatically synthesize efficient and correct low-level kernels. Evaluation across more than twenty machine learning tasks on three distinct emerging accelerators demonstrates that the approach rapidly generates high-performance kernel code—often matching or surpassing compiler-generated baselines—even for previously unseen ISAs, thereby significantly accelerating the hardware development cycle.

This work addresses the challenge of memory inefficiencies—such as redundant allocations and suboptimal usage—in large-scale software systems, which often lead to significant resource waste and performance degradation. Existing optimization approaches lack end-to-end automation and struggle to scale to codebases exceeding hundreds of millions of lines. To overcome this, we propose MOA, a novel framework that integrates multi-agent large language models with performance profiling data. MOA employs three coordinated agents—Analyzer, Checker Generator, and Patcher—to automatically detect memory anti-patterns, synthesize static checkers, and generate state-machine-guided, semantics-preserving patches. Evaluated on OpenHarmony’s C/C++ codebase (>100 million lines), MOA identified 13 memory anti-patterns (9 previously unknown), pinpointed over 10,000 inefficiency instances, and produced 769 patches with a 92.5% expert acceptance rate, reducing heap memory usage by 42.2% and binary size by 10.6% on average.

While current large language models can generate functionally correct code, they exhibit limited capability in system-level high-performance optimization and lack appropriate evaluation benchmarks. This work introduces the first executable benchmark specifically designed for assessing system-level high-performance code optimization, encompassing critical tasks such as hardware-aware optimizations, parallelization strategies, and performance bottleneck identification. The benchmark features multi-language implementations, executable validation, and runtime performance metrics to jointly evaluate both correctness and efficiency, alongside baseline and expert-optimized solutions for comparative analysis. Experimental results demonstrate that state-of-the-art models still fall significantly short of expert-level performance in tasks involving GPU operations and parallel computing, and further reveal limitations in cross-language robustness and consistency of optimization efficacy.

This work addresses the lack of automated, efficient methods for evaluating how closely existing benchmarks resemble real-world high-performance computing (HPC) applications in terms of hardware performance characteristics. The authors propose a novel performance similarity metric based on hardware usage patterns, introducing for the first time two distinct classes of computational kernels that exhibit similar performance behavior. They develop a scalable, automated evaluation framework that integrates performance feature analysis, kernel classification, and cross-platform (CPU/GPU) similarity assessment. The effectiveness and practicality of this approach are demonstrated by accurately matching computational kernels from the Kripke proxy application to those in the RAJA Performance Suite, thereby validating the method’s capability to identify functionally analogous kernels across diverse hardware architectures.

This work proposes the first modular profiling framework tailored for hardware accelerators, addressing the lack of low-overhead and flexible program analysis tools in modern computing systems. By abstracting underlying performance APIs and integrating with mainstream deep learning frameworks, the framework offers a unified interface to capture runtime events across multiple abstraction levels and enables rapid prototyping. It features a GPU-accelerated backend, multi-level event tracing, and cross-platform compatibility (NVIDIA/AMD), achieving high scalability and minimal profiling overhead in both single- and multi-GPU settings. Experimental results demonstrate that, on representative deep learning workloads, the framework achieves up to 1.3×10⁴ times faster profiling compared to conventional tools while delivering fine-grained performance insights.

This study addresses the immature compiler support for automatic vectorization on real hardware implementing the RISC-V Vector Extension (RVV 1.0), which limits its performance in scientific computing and machine learning. We present the first systematic evaluation of automatic vectorization capabilities in GCC 15 and LLVM 21 on RVV hardware, combining assembly-level microbenchmarks, perf counter calibration, and comparative experiments between manual and compiler-generated vectorization using the Qsim quantum simulator. Our analysis reveals that key performance bottlenecks—such as predicate overhead and strided memory accesses—are inadequately modeled by current cost models, while default LMUL selection is already near-optimal. Experimental results show GCC 15 outperforms LLVM 21 in four of six proxy applications; LLVM’s advantage in SGEMM/DGEMM stems from aggressive instruction reduction, highlighting both compilers’ insufficient handling of complex memory access patterns.