Data Generation for Hardware-Friendly Post-Training Quantization

2024-10-29

arXiv.org

AI Summary

Existing synthetic data generation methods for hardware-friendly zero-shot quantization (ZSQ) suffer from three critical limitations in privacy-sensitive scenarios: (1) inability to jointly optimize the full set of synthetic data, (2) neglect of data augmentation effects, and (3) output distribution shift in the final layer due to missing batch normalization (BN), severely degrading ZSQ accuracy. This paper proposes the first joint synthetic data optimization framework tailored for hardware-friendly ZSQ. Our approach: (1) incorporates augmentation-aware preprocessing and natural image prior modeling to enhance synthetic data fidelity; (2) introduces an output-feature distribution stretching loss to explicitly correct BN-induced distribution shifts; and (3) enables end-to-end, layer-wise joint optimization of synthetic images. Evaluated on classification and object detection tasks, our method achieves up to 30% absolute accuracy improvement under hardware-friendly ZSQ constraints, matching the performance of fine-tuning on real data.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Learning & Optimization for CV

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Zero-shot quantization (ZSQ) using synthetic data is a key approach for post-training quantization (PTQ) under privacy and security constraints. However, existing data generation methods often struggle to effectively generate data suitable for hardware-friendly quantization, where all model layers are quantized. We analyze existing data generation methods based on batch normalization (BN) matching and identify several gaps between synthetic and real data: 1) Current generation algorithms do not optimize the entire synthetic dataset simultaneously; 2) Data augmentations applied during training are often overlooked; and 3) A distribution shift occurs in the final model layers due to the absence of BN in those layers. These gaps negatively impact ZSQ performance, particularly in hardware-friendly quantization scenarios. In this work, we propose Data Generation for Hardware-friendly quantization (DGH), a novel method that addresses these gaps. DGH jointly optimizes all generated images, regardless of the image set size or GPU memory constraints. To address data augmentation mismatches, DGH includes a preprocessing stage that mimics the augmentation process and enhances image quality by incorporating natural image priors. Finally, we propose a new distribution-stretching loss that aligns the support of the feature map distribution between real and synthetic data. This loss is applied to the model's output and can be adapted to various tasks. DGH demonstrates significant improvements in quantization performance across multiple tasks, achieving up to a 30% increase in accuracy for hardware-friendly ZSQ in both classification and object detection, often performing on par with real data.

Problem

• Improves synthetic data for hardware-friendly quantization
• Addresses gaps in data generation and augmentation
• Enhances accuracy in zero-shot quantization tasks

Innovation

• Optimizes entire synthetic dataset jointly
• Mimics data augmentation preprocessing
• Introduces distribution-stretching loss alignment