Toward Multi-Domain and Long-Tailed Quantization via Feature Alignment and Scaling

2026-06-03

AI Summary

This work addresses the significant performance degradation of existing neural network quantization methods under multi-domain distribution shifts and long-tailed class imbalance. The authors propose a unified framework, EmaQ and its long-tailed variant EmaQ-LT, which aligns inter-domain feature distributions via cumulative distribution function (CDF) projection and enhances multi-domain quantization robustness through a sensitivity-aware weight aggregation mechanism. To tackle long-tailed imbalance, the method introduces class-conditional variance scaling and confidence-guided logit adjustment to mitigate over-confidence in majority classes. Evaluated on Office-31, Digits, and multiple long-tailed benchmarks, the proposed approach achieves state-of-the-art low-bit quantization performance, substantially outperforming current techniques.

Abstract

Quantizing deep neural networks is essential for efficient inference on resource-constrained devices. However, most existing methods are designed for single-domain and class-balanced data, leaving practical settings with domain shifts or severe class imbalance underexplored. We address these challenges with Efficient Multi-Domain Alignment Quantization (EmaQ), which aligns domain distributions through a CDF-based projection and uses sensitivity-aware weight aggregation to stabilize multi-domain quantization. We further extend EmaQ to EmaQ-LT for long-tailed quantization by introducing class-conditioned variance scaling and confidence-based logit adjustment to mitigate majority-class overconfidence. Theoretical analyses establish convergence guarantees and motivate the proposed sensitivity and scaling mechanisms. Experiments on standard, multi-domain (Office-31, Digits), and long-tailed (SynDigits-LT, CIFAR-10-LT, CIFAR-100-LT) benchmarks show that EmaQ and EmaQ-LT achieve strong low-bit performance under domain shift and class imbalance.

Problem

• quantization
• domain shift
• long-tailed
• class imbalance
• multi-domain

Innovation

• multi-domain quantization
• long-tailed learning
• feature alignment
• variance scaling
• low-bit quantization