Hybrid Compression: Integrating Pruning and Quantization for Optimized Neural Networks

2026-06-22

AI Summary

This work addresses the challenge of deploying deep neural networks on edge and embedded devices, where limited memory and computational resources necessitate a careful balance between model compression and performance. The authors propose a two-stage compression framework: first, joint pruning and quantization drastically reduce model size; second, a Mixture-of-Experts (MoE) mechanism dynamically routes inputs among multiple lightweight submodels to recover accuracy loss while preserving efficient inference. Notably, this study presents the first unified integration of pruning, quantization, and MoE architecture for effective ensemble-based compression. Experimental results demonstrate that the proposed method substantially reduces both parameter count and FLOPs of CNNs across multiple benchmark datasets, with only negligible degradation in accuracy.

Abstract

Deep neural networks have witnessed remarkable advancements in recent years and have become integral to various applications. However, alongside these developments, training and deployment of neural network models on embedding and edge devices face significant challenges due to limited memory and computational resources. These problems can be addressed with deep neural network compression, which involves a trade-off between model size and performance. In this paper, we propose a novel method for model compression through two phases. First, we utilize model compression techniques, such as pruning and quantization, to significantly reduce the model size. Then, we use Mixture of Experts to route the previously compressed models to enhance performance while maintaining a balance in inference efficiency. MoEs consist of multiple expert models (i.e., compressed models) that are moderately sized and deliver stable performance. Experimental results on several benchmark datasets show that our method successfully compresses CNN models which achieves substantial reductions in FLOPs and parameters with a negligible accuracy drop.

Problem

• model compression
• edge devices
• memory constraints
• computational resources
• performance trade-off

Innovation

• Hybrid Compression
• Pruning
• Quantization
• Mixture of Experts
• Model Compression