FOCUS: FP4 Optimization via Coupled-Relaxation and Dual-Granularity Scaling

2026-08-03

AI Summary

FP4 quantization struggles to preserve the accuracy of large language models due to its rigid coupling of quantization and dequantization scales and constraints imposed by hardware-supported discrete formats. This work proposes FOCUS, a novel framework that, for the first time, decouples these scale constraints by introducing Coupled-Relaxed Scaling (CRS) with learnable full-precision coefficients and sub-block-level Dual-Granularity Scaling (DGS). These innovations enhance model accuracy while maintaining compatibility with MXFP4/NVFP4 hardware formats. Leveraging post-training quantization, FOCUS achieves state-of-the-art FP4 performance across diverse large language models and benchmarks without incurring any additional inference overhead.

Abstract

Large language models (LLMs) achieve remarkable performance but are expensive to deploy due to their enormous size. FP4 quantization, with formats such as MXFP4 and NVFP4, offers an appealing solution with native hardware support on modern accelerators. However, maintaining accuracy under FP4 precision remains difficult. A key bottleneck lies in scale optimization: existing methods tightly couple the quantization and dequantization scales, forcing both to conform to the discrete low-precision format required by hardware, such as E8M0 in MXFP4. Yet the quantization scale is never stored and need not obey this constraint, suggesting a significant untapped optimization space. In this work, we propose FOCUS, a post-training quantization framework with end-to-end scale learning for FP4 Optimization via Coupled-Relaxation and Dual-Granularity Scaling. Coupled-Relaxation Scaling (CRS) relaxes the tight coupling between quantization and dequantization scales with a learnable full-precision coefficient, enabling more effective optimization without breaking hardware compliance. Dual-Granularity Scaling (DGS) further refines the quantization scale at a finer sub-block granularity, allowing more precise adaptation to local weight distributions. Experiments across multiple LLM families and benchmarks show that FOCUS achieves state-of-the-art FP4 accuracy under both MXFP4 and NVFP4 formats, while introducing no additional inference overhead. Code and quantized models will be released at https://github.com/tencent/AngelSlim.

Problem

• FP4 quantization
• scale optimization
• large language models
• hardware compatibility
• accuracy preservation

Innovation

• FP4 quantization
• Coupled-Relaxation Scaling
• Dual-Granularity Scaling
• post-training quantization
• large language models