forward-reflected-backward splitting

This paper addresses the problem of finding zeros of the sum of a co-coercive operator and a maximally monotone operator in real Hilbert spaces—a formulation that unifies various regression and classification tasks. To this end, we propose a novel doubly inertial forward–backward splitting algorithm, the first to incorporate two independent, tunable inertia parameters. Crucially, this design accelerates convergence and enhances numerical stability without incurring additional computational cost. Under standard assumptions of monotonicity and co-coercivity, we establish rigorous weak convergence of the generated iterates. Our theoretical analysis integrates tools from operator splitting, inertial acceleration, and monotone operator theory. Extensive experiments on benchmark regression and classification tasks demonstrate that the proposed method achieves faster convergence and higher accuracy than classical and recent forward–backward-type algorithms, delivering consistent state-of-the-art performance.

This work addresses the finite-time convergence of stochastic iterative algorithms for fixed-point equations accessible only through a noisy oracle. The authors propose a norm-independent, unified Lyapunov function framework constructed via a generalized Moreau envelope, which integrates Lyapunov stability theory with stochastic approximation analysis. This framework accommodates complex settings such as Markovian noise, seminorm contractive operators, and dissipative operators, yielding sharp non-asymptotic convergence bounds in both high-probability and mean-square senses. As a result, it provides a unified and refined finite-time convergence guarantee for a broad class of algorithms, including stochastic gradient descent, linear stochastic approximation, Q-learning, and temporal difference learning.

This work addresses the long-standing open problem of establishing global convergence for the multiplicative update iteration \( v^{(k+1)} = \mathrm{diag}((D_{v^{(k)}}^{1/2} M D_{v^{(k)}}^{1/2})^{1/2}) \) arising in private machine learning with Hadamard-product-structured regularized nuclear norm optimization. By integrating tools from matrix analysis and fixed-point theory, the paper provides the first rigorous proof that this iteration monotonically converges to the unique global optimum, thereby closing a critical theoretical gap. Furthermore, the study leverages Gemini 3 to assist in mathematical derivations, developing and distilling an effective human–AI collaborative proving strategy that offers a novel paradigm for AI-augmented formal mathematical research.

This work addresses the theoretical limitation of the Chambolle–Pock algorithm (CPA), whose convergence analysis traditionally relies on monotonicity assumptions, thereby restricting its applicability to non-monotone or weakly monotone optimization problems. Method: We introduce the novel “skew-weak Minty condition” to quantify the degree of non-monotonicity of primal–dual operators. Leveraging variational inequality theory, operator theory, and singular value analysis of linear mappings, we derive tight sufficient conditions on step sizes and relaxation parameters—imposing additional constraints in the non-monotone regime while allowing the relaxation parameter to exceed the classical upper bound of 2 under strong monotonicity. Contribution/Results: The framework unifies submonotone, cosubmonotone, and semimonotone operator classes. Constructive counterexamples verify the tightness of the derived bounds. Our results substantially broaden the theoretical foundation and practical scope of CPA, enabling rigorous convergence guarantees beyond standard monotonicity assumptions.

Existing federated learning (FL) algorithms lack a unified theoretical framework, resulting in fragmented convergence analyses and no formal comparative methodology. This paper establishes the first rigorous correspondence between FL and operator splitting theory—specifically Douglas–Rachford and Peaceman–Rachford splittings—thereby unifying mainstream FL algorithms as instances of iterative operator fixed-point updates. Crucially, it identifies stepsize selection as the decisive factor governing global convergence. Leveraging this framework, we propose a lightweight acceleration mechanism requiring no additional communication rounds, achieving improved convergence rates under both convex and nonconvex settings. We further derive novel algorithmic variants and provide general convergence guarantees applicable across diverse FL configurations. Theoretical findings are validated through comprehensive numerical experiments. Our work delivers a scalable, unified analytical toolkit and a systematic design paradigm for FL algorithm development.

Existing anchoring algorithms suffer from a lack of unified perspective due to their reliance on method-specific anchor constructions, hindering systematic understanding and generalization. This work proposes an operator-side Tikhonov regularization framework that unifies diverse anchoring methods by simply incorporating a vanishing regularizer into the base operator and running the original algorithm unchanged. The framework reveals the intrinsic commonality among methods such as Halpern iteration and extrapolated anchored gradient schemes, and naturally yields new variants. Theoretically, it recovers Halpern iteration with an $O(1/k)$ residual convergence rate, establishes a novel $O(1/\sqrt{k})$ guarantee for forward-step methods, and—under unconstrained monotone Lipschitz settings—achieves, for the first time, an $O(1/k)$ convergence rate for both the extragradient (EG) and past extragradient (PEG) methods.

This work addresses the limitations of classical iterative methods, which rely on forward error and are constrained by the condition number of the matrix. It introduces a new paradigm using backward error as the convergence criterion. The key contributions include the first proof that Richardson iteration achieves a condition-number-independent $O(1/k)$ convergence rate in backward error for any positive semidefinite linear system. Building on this, the authors design an accelerated algorithm, MINBERR, attaining an $O(1/k^2)$ convergence rate. They further integrate backward error minimization into Krylov subspace methods and extend the approach to general linear systems. The resulting general-purpose solver has complexity $O(n^2/\varepsilon)$, while MINBERR achieves $O(n^2/\sqrt{\varepsilon})$, demonstrating superior numerical performance in benchmark experiments.

This work addresses stochastic composite inclusion problems that may be non-monotone, particularly tackling the challenge of lacking effective variance reduction methods under biased estimators. The authors propose a unified framework that, for the first time, introduces biased variance-reduced estimators to inclusion and fixed-point problems, designing a new class of estimators tailored for the forward-reflected-backward splitting algorithm and providing a unified analysis covering both unbiased and biased settings. By integrating variance reduction techniques such as loopless-SVRG and SAGA, the method achieves an expected residual convergence rate of O(1/k) and almost sure convergence in the unbiased case, with oracle complexities of O(n^{2/3}ε^{-2}) and O(ε^{-10/3}), respectively. In the biased setting, the corresponding complexities are O(n^{3/4}ε^{-2}) and O(ε^{-5}). The approach is validated through applications in AUC optimization and policy evaluation in reinforcement learning.

This work addresses the challenge of computing fixed points of contractive mappings without requiring prior knowledge of the contraction factor or manual hyperparameter tuning. We propose a fully adaptive Halpern-type algorithm that operates without line searches, bisection procedures, or any user-specified parameters, automatically exploiting the inherent contractiveness of the mapping to achieve explicit linear convergence even in the absence of a priori estimates of the contraction constant. Theoretical analysis establishes linear convergence rates both in terms of fixed-point residuals and distance to the solution, with an iteration complexity of 𝒪(ε⁻¹ln(ε⁻¹)) for solving cocoercive equations. By integrating Tikhonov regularization and Nesterov acceleration, we further extend the algorithm’s applicability. Numerical experiments confirm its superiority over existing adaptive methods, offering strong theoretical guarantees alongside low computational overhead.

This study addresses the limitations of existing convergence proofs for algorithms combining sampled updates with frozen target refreshment, which typically rely on specific structural assumptions and uniformly bounded errors. By modeling sampled updates as stochastic operators, this work proposes a general contraction analysis framework that requires no gradient structure assumptions and permits errors to grow across iterations. The framework unifies the convergence bounds of deterministic frozen targets and stochastic gradients, effectively relaxing restrictions on linear approximations and uniform error boundedness. Based on this formulation, finite-time bounds are derived for arbitrary target update intervals, and geometric convergence is established. Temporal difference experiments further validate the theoretically predicted contraction rates and the scaling behavior of error lower bounds.