Problem
Research questions and friction points this paper is trying to address.
mixed-precision quantization
discrete allocation
network recovery
quadratic unconstrained binary optimization (QUBO)
Task-Aware Hybrid QUBO Optimization for Structured Neural Network Pruning
🤖 AI Summary
研究提出了一种任务感知的QUBO方法来解决混合精度量化中权重和激活位宽的离散分配问题，通过直接验证的PROTES搜索进行优化，以提高网络性能同时降低成本。
📝 Abstract
Neural network pruning can be formulated as a combinatorial optimization problem, yet many existing approaches rely on independent filter-importance scores or simplified objective functions. In this work, we propose a Hybrid Quadratic Unconstrained Binary Optimization (QUBO) framework for structured filter pruning that combines task-aware sensitivity information with interactions between candidate filters. The formulation incorporates first-order Taylor sensitivity and Weight-Fisher sensitivity into the linear component of the objective and can additionally incorporate activation similarity into the quadratic interactions. To control the target pruning cardinality without introducing an explicit quadratic cardinality penalty, we use a binary search over the capacity incentive to identify a coefficient that empirically yields the target pruning cardinality. We further investigate a two-stage QUBO--Tensor-Train refinement strategy in which the QUBO solution initializes gradient-free probabilistic black-box optimization to search for improved pruning masks using the downstream metric. Experiments on the SIDD image denoising task and a Half-UNet model show that the Hybrid QUBO achieves higher PSNR and SSIM than the evaluated Taylor and L1-based QUBO baselines at the studied pruning target. Multi-seed experiments under a fixed dataset protocol are used to assess robustness, while controlled sub-problem experiments demonstrate that Tensor-Train refinement becomes increasingly valuable as the combinatorial problem size grows. The results support Hybrid QUBO as a task-aware structured pruning framework for the evaluated setting, while also highlighting the computational and deployment limitations of mask-based pruning.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Task-aware QUBO
Mixed-precision quantization
Bit-operation cost
PROTES search
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
No related jobs found.
