Double-Precision Matrix Multiplication Emulation via Ozaki-II Scheme with FP8 Quantization

2026-03-11

AI Summary

This work addresses the challenge of performing high-precision FP64 matrix multiplication (DGEMM) on modern hardware architectures that prioritize low-precision compute capabilities—such as INT8 or FP8—at the expense of double-precision support. To bridge this gap, the study presents the first successful adaptation of the Ozaki-II algorithm to FP8 arithmetic units, introducing an efficient DGEMM emulation method that integrates FP8 quantization, Ozaki-II decomposition, and a customized low-precision matrix multiply-accumulate (MMA) scheduling strategy. The proposed approach substantially reduces the number of required FP8 matrix multiplications compared to Ozaki-I, enabling accurate FP64 results on FP8-capable GPUs while significantly lowering computational overhead and preserving numerical fidelity.

Technology Category

• Machine Learning: Hardware-aware ML
• Search and Optimization: Distributed Search
• Computer Vision: Learning & Optimization for CV

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

In high-performance computing (HPC) applications, FP64 arithmetic remains indispensable for ensuring numerical accuracy and stability. However, in recent hardware generations, improvements in FP64 arithmetic performance have been relatively modest. Consequently, achieving sustained performance gains for FP64 computations necessitates the effective utilization of high-throughput low-precision arithmetic, such as INT8 and FP8. In several recent architectures, such as NVIDIA Blackwell Ultra and NVIDIA Rubin, INT8 performance has been significantly reduced, making reliance on INT8 alone insufficient. The use of FP8 arithmetic is thus increasingly important. In this paper, we propose a method for emulating double-precision (FP64) general matrix--matrix multiplication (DGEMM), a fundamental and performance-critical kernel in many HPC applications, using FP8 matrix multiply-accumulate (MMA) units. The Ozaki-I and Ozaki-II schemes are well established as foundational approaches for emulating DGEMM via low-precision arithmetic. For DGEMM emulation via the Ozaki-I scheme, implementations using INT8, FP8, and FP16 MMA units have been proposed, all of which can be realized based on the same underlying algorithmic structure. In contrast, although implementations of DGEMM emulation via the Ozaki-II scheme using INT8 MMA units have been reported, the original algorithm cannot be directly adapted to exploit FP8 MMA units. In this work, we introduce a novel technique to overcome this limitation and demonstrate FP64 matrix multiplication emulation based on the Ozaki-II scheme that operates on FP8 MMA units. Compared to FP8-based emulation via the Ozaki-I scheme, our method significantly reduces the number of required FP8 matrix multiplications and enables efficient FP64 emulation on emerging GPU architectures.

Problem

• FP64
• FP8
• DGEMM
• Ozaki-II
• matrix multiplication

Innovation

• FP8
• Ozaki-II
• DGEMM
• matrix multiplication emulation
• low-precision arithmetic