Emulation of Complex Matrix Multiplication based on the Chinese Remainder Theorem

2025-12-09

AI Summary

This work addresses the challenge of efficiently executing high-precision (single- or double-precision) complex matrix multiplication on low-precision (INT8) hardware, which lacks native support for complex arithmetic. We propose Ozaki-II-CRT, the first Ozaki-II decomposition framework tailored for the complex domain, integrating the Chinese Remainder Theorem (CRT) and an error compensation mechanism to emulate high-precision complex GEMM on INT8 tensor cores with tunable accuracy. Our method enables fine-grained precision control, allowing flexible trade-offs between computational speed and numerical fidelity. On NVIDIA B200 GPUs, it achieves 4.0×–6.5× speedup over cuBLAS’s native complex routines—supporting both high-accuracy configurations and ultra-fast, reduced-precision execution modes. This significantly enhances the capability of low-precision accelerators to handle complex-intensive scientific computing workloads.

Technology Category

• Machine Learning: Hardware-aware ML
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Security and Privacy: Applications of cryptography
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Modern computing architectures feature low-precision matrix multiplication units that achieve substantially higher throughput than their high-precision counterparts. Motivated by this architectural trend, the emulation of high-precision matrix multiplication using low-precision hardware has attracted significant interest in the high-performance computing community. Ozaki, Uchino, and Imamura introduced the Ozaki-II scheme as a general framework for emulating matrix multiplication. Building on this framework, Uchino, Ozaki, and Imamura developed high-performance and power-efficient techniques for emulating single- and double-precision real matrix multiplication on INT8 matrix engines. Extending this line of research, the present study proposes high-performance emulation methods for single- and double-precision complex matrix multiplication on INT8 matrix engines, based on the Ozaki-II scheme. On an NVIDIA B200 GPU, the proposed methods achieve 4.0x--5.6x and 4.4x--6.5x speedups over the native single- and double-precision complex matrix multiplication routines from cuBLAS, respectively, for sufficiently large problem sizes. When lower accuracy than that of the standard routine is acceptable, the proposed methods can operate at even higher speed. Conversely, with only a modest increase in computation time, they can also deliver higher accuracy than the standard routines. These properties suggest that the proposed approach has the potential to serve as a default algorithm across a wide range of applications.

Problem

• Emulate high-precision complex matrix multiplication using low-precision INT8 hardware.
• Achieve speedups over native routines while maintaining or improving accuracy.
• Provide a flexible algorithm for applications with varying accuracy needs.

Innovation

• Emulating complex matrix multiplication using Chinese Remainder Theorem
• Leveraging low-precision INT8 hardware for high-precision computations
• Achieving speedups over native routines with adjustable accuracy