Consecutive Support Matching Induced Parameter Tuning Accelerates Momentum Iterative Hard Thresholding

2026-06-08

AI Summary

This work addresses the instability and heavy parameter tuning burden of Momentum Iterative Hard Thresholding (MIHT), which suffers from frequent support set changes due to hard thresholding. To overcome this, we propose CosMIHT, a novel method that automatically transitions from exploration to near-optimal accelerated convergence by introducing a consecutive support matching mechanism. Specifically, when the support sets of two successive iterations coincide, CosMIHT employs a lightweight power method to estimate the extreme eigenvalues of the support-restricted Gram matrix and adaptively switches to the corresponding optimal Heavy Ball parameters for acceleration. Under the restricted isometry property assumption, we establish a two-stage convergence theory for the proposed algorithm. Extensive experiments demonstrate that CosMIHT achieves significantly faster convergence than existing methods in both noiseless and noisy settings while maintaining excellent recovery accuracy, thereby validating its theoretical acceleration guarantees.

Abstract

Momentum-based acceleration of iterative hard thresholding (IHT) can dramatically speed up sparse signal recovery from linear measurements, but its effectiveness hinges on careful parameter tuning -- a task complicated by the frequent support changes inherent to hard thresholding. We propose CosMIHT(Consecutive Support Matching Induced Momentum IHT), which resolves this difficulty through a simple adaptive rule: start with the conservative parameters and whenever two consecutive iterates share the same support, estimate the extreme eigenvalues of the support restricted Gram matrix via a lightweight power method and switch to the corresponding optimal heavy-ball parameters. This mechanism allows CosMIHT to automatically interpolate between cautious MIHT-like behavior during support discovery and near-optimal accelerated convergence after support identification. Under standard restricted isometry assumptions, we develop a two-phase convergence theory. In the \emph{wandering phase}, we establish a linear contraction of the recovery error up to a noise floor and derive an explicit upper bound on the number of iterations required to identify the correct support. In the \emph{lock-in} phase, we establish that, with a randomly initialized power method based eigenvalue estimates that depend on the number of power iterations, the algorithm enjoys, with high probability, a near-optimal accelerated convergence rate akin to the heavy ball method. We corroborate the theoretical findings with extensive numerical experiments on both noiseless and noisy measurements demonstrating that CosMIHT achieves faster convergence than state-of-the-art iterative sparse recovery techniques without compromising the recovery performance.

Problem

• sparse signal recovery
• iterative hard thresholding
• parameter tuning
• support changes
• momentum acceleration

Innovation

• adaptive momentum
• support matching
• iterative hard thresholding
• heavy-ball acceleration
• sparse recovery