Stochastic Orthogonal Regularization for deep projective priors

2025-05-19

AI Summary

Deep projection priors (DPPs) lack theoretical guarantees of linear convergence under generalized projected gradient descent (GPGD) in image inverse problems. Method: We propose the first theory-driven stochastic orthogonal regularization method, which explicitly enforces near-orthogonality of the projection operator by incorporating gradient-sampling-based stochastic orthogonality constraints into the joint autoencoder-denoiser training objective. Contribution/Results: Our method rigorously satisfies the key assumption required for GPGD’s linear convergence. Experiments on canonical inverse problems—including compressive sensing and phase retrieval—demonstrate significant acceleration (2.1× average speedup), improved reconstruction accuracy (average +1.8 dB PSNR), and enhanced noise robustness. To our knowledge, this is the first regularization framework for DPPs with provable convergence guarantees, establishing a new paradigm for theoretically grounded deep prior design.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Non-convex Optimization
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Security and Privacy: Data transparency and provenance

Abstract

Many crucial tasks of image processing and computer vision are formulated as inverse problems. Thus, it is of great importance to design fast and robust algorithms to solve these problems. In this paper, we focus on generalized projected gradient descent (GPGD) algorithms where generalized projections are realized with learned neural networks and provide state-of-the-art results for imaging inverse problems. Indeed, neural networks allow for projections onto unknown low-dimensional sets that model complex data, such as images. We call these projections deep projective priors. In generic settings, when the orthogonal projection onto a lowdimensional model set is used, it has been shown, under a restricted isometry assumption, that the corresponding orthogonal PGD converges with a linear rate, yielding near-optimal convergence (within the class of GPGD methods) in the classical case of sparse recovery. However, for deep projective priors trained with classical mean squared error losses, there is little guarantee that the hypotheses for linear convergence are satisfied. In this paper, we propose a stochastic orthogonal regularization of the training loss for deep projective priors. This regularization is motivated by our theoretical results: a sufficiently good approximation of the orthogonal projection guarantees linear stable recovery with performance close to orthogonal PGD. We show experimentally, using two different deep projective priors (based on autoencoders and on denoising networks), that our stochastic orthogonal regularization yields projections that improve convergence speed and robustness of GPGD in challenging inverse problem settings, in accordance with our theoretical findings.

Problem

• Solving imaging inverse problems with deep projective priors
• Ensuring linear convergence in generalized projected gradient descent
• Improving robustness and speed in inverse problem algorithms

Innovation

• Stochastic orthogonal regularization for deep priors
• Generalized projected gradient descent with neural networks
• Improved convergence speed and robustness in inverse problems