systolic array architecture

This work addresses the inefficiency of AI ASICs—such as TPUs—optimized for low-precision computation when executing high-precision number-theoretic transforms (NTTs) required by fully homomorphic encryption (FHE). Existing approaches suffer from precision mismatches that force NTT output reconstruction off the matrix engine’s dataflow, creating a performance bottleneck. To overcome this, the paper proposes a lightweight multi-precision systolic array architecture that, for the first time, seamlessly integrates low-precision matrix multiplication and high-precision NTT reconstruction within a unified dataflow, eliminating the need for vector processor intervention and avoiding dataflow disruption. Evaluated via matrix decomposition, shift-and-add reconstruction, 7nm OpenRoad synthesis, and SCALE-Sim simulation on a 128×128 matrix engine, the design achieves at least 1.33× speedup for NTT sizes ranging from 2¹² to 2¹⁶ with negligible hardware overhead, significantly enhancing FHE execution efficiency on standard AI ASICs.

This work addresses the challenge of realizing Strassen’s theoretical speedup for matrix multiplication on hardware, particularly FPGAs. We propose a hierarchical systolic array architecture tailored for FPGA implementation. Methodologically, we introduce the first customized multi-systolic array supporting recursive Strassen expansion and establish, for the first time, a quantitative relationship between recursion depth $ r $ and DSP resource savings—achieving up to $ 1.14^r $-fold DSP reduction—while enabling systematic mapping and holistic optimization from recursion level to hardware resources. Contributions include: (1) high computational utilization for 32×32 and 24×24 matrices; (2) significant DSP reduction with negligible increase in soft logic overhead; and (3) end-to-end integration into an ML accelerator achieving state-of-the-art (SOTA) performance. This is the first hardware-level closed-loop validation demonstrating Strassen’s resource efficiency advantage.

This work addresses the challenge of efficiently supporting polynomial multiplication—a core operation in fully homomorphic encryption and post-quantum cryptography—on general-purpose AI accelerators without dedicated hardware. The authors propose MPX, a dual-mode systolic array architecture that, for the first time, reveals an intrinsic alignment between the wavefront dataflow of systolic arrays and the multiply-accumulate pattern of polynomial multiplication. This insight enables direct polynomial multiplication without requiring Number Theoretic Transform (NTT), while sharing the same hardware used for matrix multiplication. The design incurs only 20% area overhead, leaves matrix multiplication power consumption virtually unchanged, and reduces polynomial multiplication latency by more than 1.2× compared to NTT-based approaches.

Transformer models face severe acceleration bottlenecks due to their high computational demands and memory bandwidth requirements. Method: This paper proposes MatrixFlow, a hardware–software co-designed architecture featuring a novel loosely coupled systolic array and a dataflow-driven matrix multiplication mechanism, enabling system-level joint optimization of computation, data movement, and memory access. It further introduces a flexible hardware–software mapping algorithm supporting diverse models—including BERT and ViT—and validates the design via full-system gem5 simulation. Contribution/Results: Experiments show that MatrixFlow achieves up to 22× speedup over many-core CPUs, and outperforms state-of-the-art loosely coupled and tightly coupled accelerators by 5× and 8×, respectively. It significantly reduces memory overhead and improves energy efficiency.

Existing hardware multipliers struggle to efficiently support the dynamic precision requirements of mixed-precision quantized neural networks at runtime, leading to a trade-off between resource utilization and model accuracy. To address this challenge, this work proposes a runtime-reconfigurable, multi-precision, multi-channel bit-level systolic array architecture that, for the first time, enables dynamic inter-layer mixed-precision multiplication. By integrating bit-level systolic arrays, runtime reconfiguration mechanisms, and multi-channel parallel processing, the proposed architecture achieves 1.32–3.57× inference acceleration on an Ultra96 FPGA, with reduced critical-path delay and support for operating frequencies up to 250 MHz. This design significantly enhances model accuracy adaptability while maintaining high hardware efficiency.

Deploying deep neural networks on resource-constrained embedded systems presents significant challenges in simultaneously achieving energy efficiency, correctness, and security—particularly in safety-critical domains such as healthcare, where models are vulnerable to fault injection attacks and lack dynamic accuracy guarantees. To address this, this work proposes a hardware-software co-design approach. On the software side, it introduces the first adaptive precision quantization method that jointly ensures real-time operation, dynamic adjustability, and formal correctness, leveraging sensitivity analysis to stabilize decision boundaries. On the hardware side, it designs a customized systolic array supporting left-to-right, most-significant-bit (MSB)-first computation to enhance resilience against bit-flip errors. Experimental results demonstrate the feasibility of the proposed framework, with preliminary findings indicating substantial improvements in both reliability and resource efficiency for edge AI deployments.

This work addresses the inefficiency of conventional square systolic arrays in handling input-dependent and highly skewed matrices prevalent in large language models, which leads to poor hardware utilization. To overcome this limitation, the authors propose SISA, a novel architecture featuring the first scalable horizontally striped systolic array. Without increasing the number of processing units, SISA employs fine-grained partitioning and an independent scheduling mechanism to efficiently support both small or skewed matrix operations and large-scale GEMM workloads. Evaluated on representative large language model tasks, SISA achieves up to 8.52× speedup and reduces the energy-delay product by 93% compared to state-of-the-art monolithic systolic arrays of equivalent scale.

This work addresses the challenges of existing CPU matrix extensions, which struggle to efficiently adapt across diverse architectures due to high hardware/software overhead, tight coupling with the processor pipeline, and fine-grained synchronization requirements. To overcome these limitations, the authors propose a unified and configurable CPU matrix extension architecture that decouples the matrix unit from the pipeline, enabling asynchronous execution, flexible-granularity matrix-multiply abstractions, and mixed-precision computation. This design achieves low-overhead integration while maintaining synergy with existing compute and memory resources. Evaluated on four open-source CPU platforms, the approach attains over 90% GEMM utilization. It outperforms Intel AMX by 1.57× on ResNet, 1.57× on BERT, and 2.31× on Llama3. A 4 TOPS@2GHz matrix unit implemented in 14nm technology occupies only 0.53 mm².

NVIDIA Tensor Cores across V100, A100, H100, and B200 GPUs exhibit non-IEEE 754-compliant numerical behaviors—including divergent rounding modes, accumulator bit widths, normalization points, and carry handling—leading to cross-architecture computational irreproducibility. Method: We conduct the first systematic, multi-generation numerical reverse-engineering of Tensor Cores, combining targeted test-vector analysis with floating-point theory to construct a high-fidelity software simulation model. Contribution/Results: The model precisely reproduces mixed-precision matrix multiply-accumulate (MMA) operations at 8-bit, 16-bit, and 19-bit precisions, enabling hardware-level reproducible simulation. It is the first open-source, cross-architecture consistent reference tool for Tensor Core numerical behavior, bridging a critical gap in hardware-aware numerical modeling. This enables rigorous design and verification of mixed-precision algorithms, scientific computing validation, and AI system portability studies.

This paper addresses the challenge of automatically mapping PyTorch models to synthesizable hardware. We propose an open-source, end-to-end compilation toolchain that innovatively integrates Allo (an accelerator design language), Calyx (a hardware intermediate representation), and CIRCT (an LLVM-based hardware compilation framework). A key methodological contribution is a memory-partitioning compilation pass tailored for memory-intensive machine learning workloads, which significantly enhances data parallelism and on-chip memory efficiency. Our core contributions are threefold: (1) the first fully automated, synthesizable translation from PyTorch frontend to SystemVerilog RTL; (2) a memory optimization strategy that preserves functional correctness while substantially reducing off-chip bandwidth pressure; and (3) experimental validation showing that the generated FPGA implementations achieve throughput and resource utilization comparable to industrial-grade, closed-source tools such as Vitis HLS.