data layout transformation

This work addresses the performance limitations of data-intensive applications in edge computing, which often suffer from poor memory locality or excessive memory footprint. To overcome these challenges, the authors propose a hardware-software co-design that integrates a tensor memory engine into the general-purpose CPU datapath. This engine enables runtime dynamic reconfiguration of memory layouts to significantly enhance cache locality, without requiring application code modifications, offloading computation to memory, or incurring memory overhead. By preserving the CPU’s role as the primary compute unit, the approach effectively improves overall system performance. Notably, this study presents the first practical implementation of runtime memory layout reconfiguration on commercial SoC/FPGA platforms, demonstrating both innovation and real-world applicability.

To address the challenges of explicit data movement, asynchronous management, and high programming complexity in tensor computations on modern GPUs—particularly NVIDIA’s Hopper architecture—this paper introduces Cypress, a task-driven tensor programming model. Cypress introduces a novel task-level abstraction with sequential semantics, coupled with declarative memory/device mapping and fully automatic compiler scheduling. This enables coordinated offloading to asynchronous hardware units—including the Tensor Memory Accelerator (TMA) and Tensor Cores—while eliminating application-level explicit synchronization, manual data transfers, and concurrency control. Implemented atop a CUDA backend with warp-specialized kernel generation, Cypress achieves 88%–106% of cuBLAS performance on GEMM and 80%–98% of state-of-the-art implementations on Flash Attention. The model significantly improves developer productivity and hardware utilization without sacrificing performance.

This work addresses the challenge of automatically generating high-performance GPU tiled kernels from high-level tensor algebra expressions, thereby alleviating the burden of manual optimization. The authors propose an end-to-end compilation framework that integrates layer-wise lowering, expression rewriting, automated schedule search, reduction fusion, and tiling optimizations. For the first time, this framework automatically discovers high-efficiency kernels—comparable to FlashAttention-3—from the mathematical specification of attention operators, while introducing a novel scheduler that preserves program structural regularity. Evaluated on GH200 and RTX 5090 GPUs, the generated kernels achieve up to 23% and 42% higher throughput, respectively, and match or surpass hand-optimized cuDNN kernels across multiple long-sequence configurations.

In high-level synthesis (HLS), jointly optimizing code transformations, pragma insertion, and cache-blocking size selection is challenging due to tight coupling, a vast decision space, and difficulty in guaranteeing semantic correctness. Method: This paper proposes the first unified modeling framework that jointly encodes all three aspects as a single, isomorphic optimization problem—supporting “zero-transformation” decisions—and leverages HLS compiler–driven constraint derivation coupled with nonlinear programming (NLP) to automatically and correctly optimize regular loop nests. Contribution/Results: It introduces the first paradigm for co-optimizing transformations, pragmas, and blocking sizes, with built-in semantic equivalence preservation. Evaluated on multiple benchmark kernels, the approach significantly improves quality-of-results (QoR), accurately identifies cases requiring or forbidding transformations, and generates high-performance, formally verifiable optimized code.

Manual pragma configuration in high-level synthesis (HLS) suffers from low efficiency and an exponentially large search space. Method: This paper proposes the first nonlinear programming (NLP)-based automated pragma insertion framework, jointly optimizing loop-level pragmas—including pipelining, function unit replication, and data caching. It innovatively models discrete pragma configurations as continuous, differentiable variables and constructs analytical performance/resource models with theoretical lower-bound guarantees, solved globally via NLP. Integrated with pragma semantic analysis and the Merlin compiler, and augmented by design-space pruning, the framework explores billion-scale configurations within seconds to minutes. Contribution/Results: Experimental evaluation shows kernel performance approaching hand-tuned implementations, resource estimation error <8%, and latency lower-bound error ≤12%.

This work addresses the challenges of excessive search space and difficulty in bottleneck identification when large language models (LLMs) automatically generate GPU kernels, stemming from a mismatch between the granularity of optimization knowledge and LLM reasoning. To bridge this gap, we propose the Hierarchical Transfer-Aware Memory (HTAM) framework, which organizes optimization knowledge into a two-level hierarchical transfer graph encompassing coarse-grained directions and fine-grained strategies. HTAM guides LLMs in generating efficient CUDA code through a state-aware mechanism that jointly selects global optimization directions and retrieves relevant local strategies. Evaluated on the full KernelBench suite, HTAM significantly improves correctness rate, the proportion of fast solutions, and speedup, while demonstrating strong generalization across backend configurations and the Robust-KBench benchmark.

Modern GPU programming faces a fundamental trade-off between abstraction level and hardware control: excessive abstraction impedes performance optimization, while overly low-level approaches impose significant development burdens. This work proposes TLX, an extension to the Triton language based on a Multi-Instruction, Multi-Warp (MIMW) execution model—the first to integrate MIMW into a high-level GPU programming framework. TLX operates at the warp-group granularity, explicitly supporting multi-warp scheduling, shared memory orchestration, asynchronous operations, and cluster-aware control flow. It preserves Triton’s elegant block-level programming model while enabling efficient exploitation of native hardware features. Experimental results demonstrate that TLX kernels achieve state-of-the-art performance with substantially reduced development effort and have been successfully deployed in large-scale training and inference systems.

This work proposes CuTe, a novel mathematical framework for representing and manipulating hierarchical tensor layouts through a layout algebra that supports operations such as concatenation, tiling, and inversion. Modern high-performance computing and deep learning rely heavily on specialized tensor instructions whose performance and correctness are critically dependent on intricate, hardware-specific data layouts. CuTe enables unified compile-time derivation, verification, and thread mapping of these layouts, significantly simplifying GPU kernel development while supporting expressive, general-purpose tensor transformations. The framework has been integrated into production systems including the NVIDIA CUTLASS library and the CuTe DSL, effectively bridging the gap between hardware constraints and software flexibility.

This work addresses the limited optimization of data movement in existing DNN compilers, which typically target only a subset of operators and struggle with redundant transfers in large models. To overcome this, the authors propose VTC, a novel framework that introduces the concept of virtual tensors, replacing physical data movement with index mapping and integrating a new data-movement elimination algorithm to uniformly optimize across the full spectrum of data-movement operators. VTC enables seamless optimization for arbitrary tensor operator compositions and interoperates effectively with existing compute kernels. Evaluated on NVIDIA GPUs, VTC achieves an average speedup of 1.28× (up to 1.93×) and reduces inference memory consumption by 17.5% on average (up to 60%).

This work addresses the high barrier to entry in manually developing high-performance tensor kernel functions for AI accelerators, a process that traditionally demands deep expertise in tiling strategies, instruction selection, data layout optimization, and operator fusion. To alleviate this burden, the paper proposes an automated synthesis approach that eliminates the need for handcrafted rules by integrating program synthesis, SMT solving, and algebraic transformations of computation graphs. The method formally verifies semantic equivalence over unbounded tensors and systematically explores the space of tiling strategies and instruction/operator fusion under hardware constraints. This enables the automatic generation of kernels that are not only semantically correct but also highly efficient, significantly reducing memory traffic and improving execution performance while substantially lowering the development effort required.