admm for tensor completion

This study addresses the problem of partially observed low-rank tensor completion, a high-dimensional generalization of matrix completion. Building upon the nuclear norm minimization framework, the authors propose an improved adaptive alternating direction method of multipliers (ADMM) algorithm that incorporates an over-relaxation mechanism and a dynamic penalty parameter update strategy. This formulation efficiently decomposes the original problem into subproblems amenable to parallel computation and leverages closed-form proximal operators to enable rapid iterations. The proposed method achieves significantly accelerated convergence and enhanced completion accuracy, outperforming state-of-the-art approaches in terms of normalized mean squared error (NMSE). Furthermore, when combined with an advanced initialization strategy, both its performance and convergence speed are further improved.

This work addresses the computational challenge in traditional alternating direction method of multipliers (ADMM) for bilinear minimax (saddle-point) optimization problems, where evaluating complex proximal operators is often required. The authors propose a novel ADMM variant that decomposes the original problem into two alternating substeps: a generalized projection onto the constraint set \( S \) and a Euclidean projection onto the set \( C \). The key innovation lies in the exact reformulation—without approximation or linearization—of the ADMM proximal operator under the bilinear structure into a computable generalized projection. By integrating tools from convex analysis and projection techniques, the method establishes a provably convergent and computationally efficient framework, significantly simplifying the solution process for bilinear minimax problems.

This work studies the convergence of power iteration for decomposing random overcomplete tensors (where rank far exceeds dimension). Prior conjectures suggested logarithmic convergence; we establish the first tight polynomial lower bound on the required number of iterations, proving that polynomial—rather than logarithmic—steps are necessary. Methodologically, we introduce a novel analytical framework based on Gaussian conditioning, which overcomes fundamental limitations of traditional approximate message passing (AMP) analyses—specifically, their reliance on proportional limits and bounded iteration counts. Integrating tools from random matrix theory and rigorous monotonicity analysis, we prove strict monotonic increase of the objective function throughout iterations. Empirical results confirm successful recovery of true components within polynomial time. This work provides the first precise characterization of the computational complexity of power iteration for overcomplete tensor decomposition, thereby establishing a foundational theoretical basis for high-dimensional tensor learning.

Weighted Low-Rank Approximation (WLRA) seeks a rank-$k$ matrix $XY^ op$ minimizing the weighted Frobenius norm $|W circ (M - XY^ op)|_F$, given a matrix $M$ and a nonnegative weight matrix $W$. This problem is NP-hard and hard to approximate. This paper proposes the first alternating minimization framework for WLRA that simultaneously achieves strong theoretical guarantees and high efficiency. Our method integrates a high-accuracy multi-response regression solver into each alternating update step, enabling approximate yet controllable subproblem solving. Crucially, it preserves global convergence while reducing the per-iteration time complexity from $O(|W|_0 k^2)$ to $O(|W|_0 k)$, where $|W|_0$ denotes the number of nonzero entries in $W$—yielding substantial speedups for sparse weighting patterns. Experiments demonstrate state-of-the-art performance on matrix completion and noise-robust recovery tasks.

This work studies rank-1 symmetric matrix completion, focusing on the global convergence of gradient descent (GD) under small random initialization. Theoretically, we prove that GD converges globally to the true low-rank solution in logarithmic iterations—provided the initialization norm satisfies an explicit upper bound inversely proportional to the number of observed entries—without requiring tailored initialization schemes or explicit regularization. This is the first rigorous nonconvex analysis establishing end-to-end global convergence for “small initialization + GD” in matrix completion. It reveals the critical role of GD’s implicit regularization in constraining optimization trajectories, escaping saddle points, and avoiding spurious local minima. Moreover, we quantify the interplay among sample complexity, initialization scale, and convergence rate, providing a new theoretical foundation for understanding the implicit bias of first-order methods in low-rank inverse problems.

This work proposes an online learning strategy for accelerating the convergence of the Alternating Direction Method of Multipliers (ADMM) when solving structured convex optimization problems—such as time-varying quadratic programs arising in model predictive control—by dynamically tuning its relaxation parameter. The approach targets scenarios where the problem structure remains fixed but parameters change over time, thereby circumventing the need for costly matrix refactorizations typically required in conventional penalty parameter adjustment. For the first time, convergence guarantees are established for ADMM with time-varying penalty and relaxation parameters. By integrating ideas from reinforcement learning into parameter scheduling, the method achieves substantial improvements in solution efficiency while maintaining low computational overhead. Implemented within the OSQP framework, the proposed strategy significantly reduces both iteration counts and actual solve times on standard quadratic programming benchmarks.

Existing end-to-end approaches to solving constrained convex optimization problems often fail to strictly satisfy constraints and lack guarantees of optimality. This work proposes a trainable architecture based on unfolded ADMM that enforces hard constraints through an embedded constraint-satisfaction module and a differentiable equality-constraint correction layer, ensuring exact feasibility at every iteration. Furthermore, first-order optimality conditions are incorporated as soft constraints into the training objective to guide convergence toward high-quality solutions. The proposed method uniquely unifies strict constraint satisfaction with optimality-aware learning within an unfolded optimization framework. Empirical results across multiple constrained convex optimization tasks demonstrate substantial improvements over conventional black-box end-to-end models, achieving both high solution accuracy and strong constraint compliance.

This work addresses the challenges of manifold mismatch and lack of convergence guarantees when directly embedding score-based generative models into optimization algorithms such as ADMM. To resolve these issues, the authors propose the ADMM-PnP framework, which for the first time enables a Plug-and-Play method within ADMM with provable convergence. The framework introduces an AC-DC three-stage denoising mechanism that integrates additive Gaussian noise with self-correction (AC), direction-corrected conditional Langevin dynamics (DC), and score-based denoising, while combining constant and adaptive stepsize strategies. This design effectively mitigates manifold mismatch and simultaneously ensures geometric consistency and algorithmic convergence. Experiments demonstrate that the proposed method consistently achieves superior solution quality over existing baselines across various inverse problems, empirically validating its theoretical convergence guarantees.

This work addresses the low-rank tensor completion problem by proposing a novel non-convex regularizer—the ratio of the tensor nuclear norm to the Ky Fan $p$-$k$ norm (TNPK)—to more accurately approximate the tensor tubal rank. The proposed regularizer enjoys scale invariance, parameter flexibility, and admits a closed-form solution under certain conditions, with the tensor nuclear norm (TNK) and tensor Ky Fan norm (TNF) as its special cases. Leveraging properties of the tensor null space, the authors theoretically establish that low-rank tensors correspond to local minima of the proposed model. For optimization, they design a proximal operator based on the inverse Ky Fan $p$-$k$ norm and integrate it into an ADMM algorithm with guaranteed subsequential convergence. Extensive experiments demonstrate that the method significantly outperforms state-of-the-art approaches on both synthetic and real-world datasets.

This work addresses the high computational cost and inefficiency of existing methods in low-Tucker-rank tensor sensing, particularly when handling high-dimensional multimodal data. To overcome these limitations, we propose a stochastic alternating minimization algorithm that directly optimizes over the core tensor and factor matrices in the Tucker decomposition. By avoiding expensive full-tensor projection operations and enabling mini-batch gradient updates on low-dimensional factor matrices, our approach significantly reduces computational overhead. As the first to introduce stochastic alternating minimization into Tucker-structured tensor sensing, this method breaks the reliance of prior approaches on full gradients or full-tensor operations, achieving provable convergence while markedly improving efficiency. Experimental results on synthetic data demonstrate that our algorithm attains substantially faster convergence than state-of-the-art stochastic tensor recovery baselines under identical runtime constraints.