IPPRO: Importance-based Pruning with PRojective Offset for Magnitude-indifferent Structural Pruning

2026-04-11

AI Summary

Conventional magnitude-based criteria in structured pruning often fail to identify redundant filters, leading to suboptimal compression. To address this, we propose IPPRO—a projection-space gradient dynamics analysis method for filter importance estimation. IPPRO abandons reliance on weight magnitudes and instead models the gradient descent trajectory in a linearly projected low-dimensional space, where it quantifies each filter’s directional contribution to optimization. Crucially, it introduces the PROscore, an amplitude-agnostic metric that enables fair and discriminative redundancy assessment. Extensive experiments across multiple CNN architectures (e.g., ResNet, VGG) and datasets (e.g., ImageNet, CIFAR-10/100) demonstrate that IPPRO achieves near-lossless compression at comparable sparsity levels—significantly reducing accuracy degradation over baselines. After fine-tuning, pruned models consistently outperform state-of-the-art structured pruning methods, validating both the effectiveness and generalizability of IPPRO’s importance evaluation mechanism.

Technology Category

Machine Learning: Learning on the Edge & Model CompressionComputer Vision: Learning & Optimization for CVSearch and Optimization: Non-convex Optimization

Application Category

Security and Privacy: Large-scale security measurementsGraph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphsWeb Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

With the growth of demand on neural network compression methods, the structured pruning methods including importance-based approach are actively studied. The magnitude importance and many correlated modern importance criteria often limit the capacity of pruning decision, since the filters with larger magnitudes are not likely to be pruned if the smaller one didn't, even if it is redundant. In this paper, we propose a novel pruning strategy to challenge this dominating effect of magnitude and provide fair chance to each filter to be pruned, by placing it on projective space. After that, we observe the gradient descent movement whether the filters move toward the origin or not, to measure how the filter is likely to be pruned. This measurement is used to construct PROscore, a novel importance score for IPPRO, a novel importance-based structured pruning with magnitude-indifference. Our evaluation results shows that the proposed importance criteria using the projective space achieves near-lossless pruning by reducing the performance drop in pruning, with promising performance after the finetuning. Our work debunks the ``size-matters'' myth in pruning and expands the frontier of importance-based pruning both theoretically and empirically.

Problem

• Challenges magnitude dominance in filter pruning decisions
• Introduces projective space for fair filter pruning evaluation
• Achieves near-lossless pruning with reduced performance drop

Innovation

• Projects filters onto projective space for pruning
• Uses gradient descent movement to measure importance
• Introduces PROscore for magnitude-indifferent pruning