MASQuant: Modality-Aware Smoothing Quantization for Multimodal Large Language Models

2026-03-05

AI Summary

Existing post-training quantization methods for multimodal large language models often suffer from performance instability due to misaligned smoothing and the lack of cross-modal computational invariance. This work proposes a novel quantization framework that effectively aligns activation distributions across modalities while preserving their low-rank structural consistency. The approach integrates a modality-aware smoothing (MAS) mechanism with a cross-modal compensation strategy based on singular value decomposition (SVD) whitening. By jointly addressing activation misalignment and structural distortion, the method achieves the first unified and stable low-bit quantization for multimodal large models. It sets a new state-of-the-art in post-training quantization performance across both dual- and tri-modal settings, demonstrating robustness and scalability without requiring retraining or extensive calibration.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Post-training quantization (PTQ) with computational invariance for Large Language Models~(LLMs) have demonstrated remarkable advances, however, their application to Multimodal Large Language Models~(MLLMs) presents substantial challenges. In this paper, we analyze SmoothQuant as a case study and identify two critical issues: Smoothing Misalignment and Cross-Modal Computational Invariance. To address these issues, we propose Modality-Aware Smoothing Quantization (MASQuant), a novel framework that introduces (1) Modality-Aware Smoothing (MAS), which learns separate, modality-specific smoothing factors to prevent Smoothing Misalignment, and (2) Cross-Modal Compensation (CMC), which addresses Cross-modal Computational Invariance by using SVD whitening to transform multi-modal activation differences into low-rank forms, enabling unified quantization across modalities. MASQuant demonstrates stable quantization performance across both dual-modal and tri-modal MLLMs. Experimental results show that MASQuant is competitive among the state-of-the-art PTQ algorithms. Source code: https://github.com/alibaba/EfficientAI.

Problem

• Post-training quantization
• Multimodal Large Language Models
• Smoothing Misalignment
• Cross-Modal Computational Invariance

Innovation

• Modality-Aware Smoothing
• Cross-Modal Compensation
• Post-Training Quantization
• Multimodal Large Language Models
• SVD Whitening