🤖 AI Summary
This work addresses the solution of general linear systems via the Randomized Block Kaczmarz (RBK) method, focusing on convergence analysis and performance enhancement. We propose a unified non-expansive block Kaczmarz framework that, for the first time, establishes rigorous convergence theory for diverse static random sampling strategies. By leveraging concentration inequalities and scale-invariant analysis techniques, we derive tight, expectation-based linear convergence rate bounds—substantially improving upon existing results. These bounds more accurately characterize the practical convergence behavior of block methods while maintaining computational efficiency even with low per-iteration cost. Extensive numerical experiments validate both the tightness of the theoretical bounds and the practical efficacy of the proposed algorithm.
📝 Abstract
To conduct a more in-depth investigation of randomized solvers for general linear systems, we adopt a unified randomized batch-sampling Kaczmarz framework with per-iteration costs as low as cyclic block methods, and develop a general analysis technique to establish its convergence guarantee. With concentration inequalities, we derive new expected linear convergence rate bounds. The analysis applies to any randomized non-extended block Kaczmarz methods with static stochastic samplings. In addition, the new rate bounds are scale-invariant which eliminate the dependence on the magnitude of the data matrix. In most experiments, the new bounds are significantly tighter than existing ones and better reflect the empirical convergence behavior of block methods. Within this new framework, the batch-sampling distribution, as a learnable parameter, provides the possibility for block methods to achieve efficient performance in specific application scenarios, which deserves further investigation.