🤖 AI Summary
Existing open-source RTL dot-product units rely on discrete arithmetic units, resulting in low throughput and poor hardware resource utilization. To address this, this paper proposes a configurable, mixed-precision dot-product unit tailored for GPGPU tensor computation. Our approach unifies floating-point and integer arithmetic pipelines, natively supporting multiplication across FP16, BF16, FP8, BF8, INT8, and UINT4 formats, while performing high-precision accumulation in FP32 or INT32. The architecture is extensible to future custom data formats. Implemented as an extension to the RISC-V Vortex GPGPU tensor core and synthesized on an AMD Xilinx Alveo U55C FPGA, the design achieves a 306.6 MHz operating frequency, a 4-cycle latency, and a peak throughput of 9.812 GFLOPS under full pipelining. Experimental results demonstrate substantial improvements in computational efficiency and hardware resource utilization for deep learning workloads.
📝 Abstract
Efficient mixed-precision MMA operations are critical for accelerating Deep Learning workloads on GPGPUs. However, existing open-source RTL implementations of inner dot products rely on discrete arithmetic units, leading to suboptimal throughput and poor resource utilization. To address these challenges, we propose a scalable mixed-precision dot product unit that integrates floating-point and integer arithmetic pipelines within a singular fused architecture, implemented as part of the open-source RISC-V based Vortex GPGPU's Tensor Core Unit extension. Our design supports low-precision multiplication in (FP16/BF16/FP8/BF8/INT8/UINT4) formats and higher-precision accumulation in (FP32/INT32), with an extensible framework for adding and evaluating other custom representations in the future. Experimental results demonstrate 4-cycle operation latency at 306.6 MHz clock frequency on the AMD Xilinx Alveo U55C FPGA, delivering an ideal filled pipeline throughput of 9.812 GFLOPS in a 4-thread per warp configuration.