Deep Network Pruning: A Comparative Study on CNNs in Face Recognition

2024-05-28

arXiv.org

AI Summary

To address the challenge of excessive model size hindering deployment of CNNs for mobile face authentication, this paper proposes a filter-level structured pruning method based on Taylor expansion, and conducts progressive compression across diverse architectures—SqueezeNet, MobileNetV2, and ResNet50. It presents the first systematic validation of Taylor-based pruning’s efficacy and universality across multi-complexity face recognition models. Crucially, it identifies an “over-dimensional” phenomenon in mainstream CNNs: high-dimensional output-channel filters are more readily pruned. Leveraging iterative pruning coupled with fine-tuning recovery, the approach achieves over 60% parameter reduction and compresses model size to 30–40% of the original, while incurring only a marginal accuracy drop of 0.5–1.2%. These results significantly satisfy the stringent latency and memory constraints of real-time mobile deployment.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Non-convex Optimization

Application Category

• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

The widespread use of mobile devices for all kind of transactions makes necessary reliable and real-time identity authentication, leading to the adoption of face recognition (FR) via the cameras embedded in such devices. Progress of deep Convolutional Neural Networks (CNNs) has provided substantial advances in FR. Nonetheless, the size of state-of-the-art architectures is unsuitable for mobile deployment, since they often encompass hundreds of megabytes and millions of parameters. We address this by studying methods for deep network compression applied to FR. In particular, we apply network pruning based on Taylor scores, where less important filters are removed iteratively. The method is tested on three networks based on the small SqueezeNet (1.24M parameters) and the popular MobileNetv2 (3.5M) and ResNet50 (23.5M) architectures. These have been selected to showcase the method on CNNs with different complexities and sizes. We observe that a substantial percentage of filters can be removed with minimal performance loss. Also, filters with the highest amount of output channels tend to be removed first, suggesting that high-dimensional spaces within popular CNNs are over-dimensionated.

Problem

• Optimizing face recognition CNNs for mobile deployment
• Reducing network size via pruning with minimal accuracy loss
• Evaluating pruning effectiveness across diverse CNN architectures

Innovation

• Network pruning via Taylor scores
• Iterative removal of less important filters
• Testing on SqueezeNet, MobileNetv2, ResNet50