register blocking

This work addresses resource and power constraints of FPGA-based neural network accelerators in edge AI scenarios, focusing on two core design challenges: dataflow architecture and tensor tiling strategies. Methodologically, it introduces an extended dataflow taxonomy—including the novel “No-Local-Reuse” paradigm—and establishes a unified evaluation matrix enabling, for the first time, cross-dimensional comparison of heterogeneous open-source accelerators (e.g., FINN/R, FlightLLM, SSR). Leveraging MAESTRO/Timeloop modeling, it integrates multi-level memory optimization (registers, LUTRAM, BRAM, HBM) with loop unrolling, pipelining, and sparsity-aware transformations to derive automated HLS/DSL/RTL design heuristics. A standardized benchmark is proposed, covering platform compatibility, accuracy, throughput, resource utilization, and energy efficiency. The contributions include a theoretical framework and practical guidelines for edge AI–FPGA co-design, identifying partial reconfiguration, hybrid dataflows, and domain-specific compilers as three key evolutionary directions.

AI kernel development faces challenges including complex hardware adaptation and insufficient expressiveness and usability of domain-specific compilers. This paper proposes a composable tiling programming model that—uniquely—decouples dataflow from the scheduling space (thread mapping, memory layout, tensorization, and pipelining). By introducing a unified block-thread abstraction, lightweight scheduling primitive annotations, and dataflow-driven, hardware-aware compilation, our approach jointly optimizes developer productivity and kernel performance. The method bridges the gap between expressive power and engineering practicality in domain-specific compilers. Evaluated on mainstream accelerators—including GPUs and ASICs—our generated AI compute kernels achieve state-of-the-art performance across key workloads, while reducing development cycles significantly. The framework delivers both flexibility in algorithmic expression and high execution efficiency.

This work addresses the performance limitations of data-intensive applications in edge computing, which often suffer from poor memory locality or excessive memory footprint. To overcome these challenges, the authors propose a hardware-software co-design that integrates a tensor memory engine into the general-purpose CPU datapath. This engine enables runtime dynamic reconfiguration of memory layouts to significantly enhance cache locality, without requiring application code modifications, offloading computation to memory, or incurring memory overhead. By preserving the CPU’s role as the primary compute unit, the approach effectively improves overall system performance. Notably, this study presents the first practical implementation of runtime memory layout reconfiguration on commercial SoC/FPGA platforms, demonstrating both innovation and real-world applicability.

This work addresses the challenge of efficiently exploiting parallelism and mitigating latency from hierarchical memory and explicit data movement in edge AI kernels deployed on resource-constrained devices. Building upon the MLIR compilation framework and leveraging kernels generated by Triton/Inductor, the study systematically evaluates three compiler-level optimizations: vectorization (Vec), hardware context-level multithreading (MT), and ping-pong double buffering (DB). Through a novel ablation staircase methodology, the authors uniquely isolate and quantify the individual and synergistic performance contributions of these techniques under varying compute-to-memory intensity regimes. The findings reveal that vectorization primarily accelerates bandwidth-bound kernels, multithreading yields significant gains after amortizing scheduling overhead, and double buffering provides additional speedup when computation and data transfer can be effectively overlapped.

Spatial dataflow architectures suffer from low programmability and heavy reliance on vendor-provided hand-optimized libraries, hindering realization of their high performance and energy efficiency potential. Method: This paper proposes TL, an end-to-end compilation framework built atop MLIR. TL introduces the first hardware-aware unified intermediate representation (IR) supporting cross-architecture modeling of topology, storage, and computation. It transcends single-tile optimization by enabling holistic co-optimization of global tile distribution, on-chip network (NoC) communication, and distributed memory reuse. Contribution/Results: By integrating hardware topology modeling, multi-level memory hierarchy analysis, and NoC-aware scheduling, TL significantly improves data reuse and reduces communication overhead on heterogeneous spatial accelerators. It achieves, for the first time, automatic and efficient mapping of high-level tiled programs (e.g., Triton) without dependence on vendor-specific libraries.

Current GPU programming models lack expressiveness for chiplet-level locality and synchronization, leading to redundant memory accesses and poor cache utilization when executing memory-intensive workloads such as large language model (LLM) inference on multi-chiplet GPUs. This work proposes Fleet, the first multi-level task programming model that explicitly exposes the chiplet hierarchy. Fleet introduces a chiplet-task abstraction that binds computation and data to specific chiplets and integrates persistent kernels, cooperative weight tiling, and per-chiplet scheduling to enable L2 cache reuse and efficient coordinated execution. Evaluated on an AMD MI350 running Qwen3-8B, Fleet reduces decoding latency by 1.3–1.5× for small batches and cuts HBM traffic by up to 37% under large batches, significantly improving L2 hit rates and achieving overall speedups of 1.27–1.30×.

Transformer inference on hardware accelerators is often bottlenecked not by computational capacity but by paged data movement and interconnect bandwidth. This work proposes a system-accelerator co-design that replaces large on-chip SRAM with small caches and a paged streaming scheduler, enabling explicit overlap of computation and data transfer through a DMA-compute-DMA-out pipeline and 4KB-tiled matrix multiplication on the loosely coupled systolic array MatrixFlow. Evaluated using an extended Gem5-AcceSys full-system simulation framework, the proposed approach achieves up to 22× speedup over a CPU-only baseline and outperforms existing loosely and tightly coupled accelerators by 5–8×. Notably, it attains 80% of the performance achievable with on-chip HBM while operating under standard PCIe host memory constraints.

This work addresses the limitations of existing sparse general matrix-matrix multiplication (SpGEMM) accelerators, which rely on static dataflows that struggle to simultaneously achieve high data reuse and load balance. To overcome these inherent constraints, we propose SegFold, the first SpGEMM accelerator architecture incorporating fine-grained dynamic scheduling and dynamic remapping of partially completed work into its dataflow design. SegFold employs a customized memory controller to exploit data reuse within local windows and integrates dynamic task distribution with a reconfigurable merging network to ensure balanced workload allocation across processing units. Experimental results demonstrate that SegFold achieves an average speedup of 1.95× over state-of-the-art SpGEMM accelerators and up to 5.3× improvement compared to the best static configuration, across a wide range of sparsity patterns and matrix sizes.

This work addresses the conflict between conventional page-granularity interleaved memory layouts and the locality demands of GEMM operations on chiplet-based GPUs, which incurs substantial remote HBM access overhead. To resolve this, the authors propose Chiplet-Contiguous Layout—a hardware- and OS-agnostic global memory organization that colocates each chiplet’s local data contiguously, thereby enabling locality-aware scheduling while remaining compatible with standard 4KB page management. This approach is the first to unify data locality optimization for GEMM with page-granular memory management in both large language model inference and training. Experiments demonstrate that, compared to 4KB interleaved layouts, the proposed method reduces remote HBM traffic by 24.7× and 19.2× on Qwen-3 30B and Llama-3.1 70B, respectively, and further achieves 4.1× and 2.1× reductions over coarse-grained locality-aware placements.