🤖 AI Summary
This work addresses the lack of theoretical guarantees for local convergence in existing physical learning methods—such as Equilibrium Propagation (EP) and Coupled Learning (CL)—when applied to linear circuit training in the small-perturbation limit. The authors propose a unified convergence analysis framework, introducing for the first time a coercivity condition based on the network’s interconnection structure, formulated as a matrix rank condition. Under this condition, they prove exponential decay of the loss function and convergence of parameters to the solution manifold. Using Sard’s theorem, they further show that this coercivity condition holds for almost all target outputs, demonstrating that degeneracies caused by symmetry are non-generic. The analysis also clarifies that both EP and the newly proposed Adjoint Coupled Learning implement exact gradient descent on the natural loss, whereas standard CL incurs third-order correction terms. Theoretical findings are validated through a kite-circuit example.
📝 Abstract
Physical learning methods train physical networks to perform computational tasks using only local update rules, exploiting the physics of the system to handle the global transfer of information. We provide the first local convergence analysis of three such methods -- Equilibrium Propagation (EP), Coupled Learning (CL), and a new method we call Adjoint Coupled Learning (AL) -- for linear circuits, in the limit of small-nudging for both discrete and continuous time. EP and AL perform gradient descent on a natural loss function, while CL follows modified dynamics with an additional cubic correction. Assuming the existence of a solution, we identify a coercivity condition, expressed as a rank condition on a matrix built from the network's incidence structure, under which the training loss decays exponentially and the parameters converge to the solution manifold. We show that coercivity can fail by exhibiting a kite circuit in which a symmetry causes the coercivity constant to degenerate on the solution manifold, but prove using Sard's theorem that such degeneracies are non-generic: coercivity holds at every point of the solution manifold for almost every choice of desired output.