🤖 AI Summary
This paper addresses the noiseless complete dictionary learning problem—exactly recovering a square, complete dictionary from observed signals and achieving sparse representations. To overcome key limitations of existing heuristic algorithms—including lack of theoretical guarantees and poor scalability in large-scale settings—we propose the first nonconvex discrete dictionary learning framework with provably linear convergence. Our approach introduces a novel preprocessing technique that enforces near-orthogonality of the dictionary, substantially relaxing convergence conditions. By integrating this preprocessing with alternating minimization, we further extend the framework to mini-batch and online learning paradigms. Theoretical analysis is rigorous and comprehensive. Empirical evaluations on both synthetic and real-world datasets demonstrate superior performance over state-of-the-art methods, achieving high-fidelity dictionary recovery, strong scalability, and real-time adaptability.
📝 Abstract
This paper focuses on the noiseless complete dictionary learning problem, where the goal is to represent a set of given signals as linear combinations of a small number of atoms from a learned dictionary. There are two main challenges faced by theoretical and practical studies of dictionary learning: the lack of theoretical guarantees for practically-used heuristic algorithms and their poor scalability when dealing with huge-scale datasets. Towards addressing these issues, we propose a simple and efficient algorithm that provably recovers the ground truth when applied to the nonconvex and discrete formulation of the problem in the noiseless setting. We also extend our proposed method to mini-batch and online settings where the data is huge-scale or arrives continuously over time. At the core of our proposed method lies an efficient preconditioning technique that transforms the unknown dictionary to a near-orthonormal one, for which we prove a simple alternating minimization technique converges linearly to the ground truth under minimal conditions. Our numerical experiments on synthetic and real datasets showcase the superiority of our method compared with the existing techniques.
