Exploring Neural Network Pruning with Screening Methods

2025-02-11

AI Summary

To address the conflict between high computational overhead of deep neural networks (DNNs) and stringent resource constraints on edge devices, this paper proposes a dual-granularity pruning framework grounded in statistical significance analysis. The method introduces the first cross-class component significance modeling, integrating connection-level and channel-level weighted selection to enable joint unstructured and structured pruning; multi-class sensitivity evaluation further enhances pruning accuracy and hardware compatibility. On vision benchmarks, FNN/CNN models achieve 40–60% compression with <1.2% accuracy degradation. In three comparative experiments, it outperforms state-of-the-art (SOTA) pruning methods in two cases. The core contribution is a novel significance-driven pruning paradigm that jointly optimizes pruning efficiency, structural deployability, and classification robustness—thereby advancing practical DNN deployment on resource-constrained edge platforms.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Other Foundations of Computer Vision
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Security and Privacy: Large-scale security measurements
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Deep neural networks (DNNs) such as convolutional neural networks (CNNs) for visual tasks, recurrent neural networks (RNNs) for sequence data, and transformer models for rich linguistic or multimodal tasks, achieved unprecedented performance on a wide range of tasks. The impressive performance of modern DNNs is partially attributed to their sheer scale. The latest deep learning models have tens to hundreds of millions of parameters which makes the inference processes resource-intensive. The high computational complexity of these networks prevents their deployment on resource-limited devices such as mobile platforms, IoT devices, and edge computing systems because these devices require energy-efficient and real-time processing capabilities. This paper proposes and evaluates a network pruning framework that eliminates non-essential parameters based on a statistical analysis of network component significance across classification categories. The proposed method uses screening methods coupled with a weighted scheme to assess connection and channel contributions for unstructured and structured pruning which allows for the elimination of unnecessary network elements without significantly degrading model performance. Extensive experimental validation on real-world vision datasets for both fully connected neural networks (FNNs) and CNNs has shown that the proposed framework produces competitive lean networks compared to the original networks. Moreover, the proposed framework outperforms state-of-art network pruning methods in two out of three cases.

Problem

• Reducing DNN computational complexity
• Pruning non-essential network parameters
• Enabling deployment on resource-limited devices

Innovation

• Neural network pruning framework
• Statistical analysis significance
• Unstructured and structured pruning