Optimization Algorithm Design via Electric Circuits

2024-11-04

arXiv.org

AI Summary

Conventional design of convex optimization algorithms is often ad hoc and lacks systematic principles. Method: This paper proposes a novel algorithm construction paradigm grounded in RLC circuit modeling: (i) formulate a continuous-time circuit dynamical system whose trajectories converge to the optimizer; (ii) apply automated symbolic discretization coupled with Lyapunov stability analysis to rigorously guarantee global convergence of the resulting discrete-time iterative algorithm. Contribution/Results: This work establishes the first systematic mapping from circuit physics to optimization algorithm design, enabling provably convergent translation from continuous dynamics to discrete algorithms. It uniformly reconstructs classical methods—including gradient descent and Nesterov’s accelerated gradient—and synthesizes multiple new variants, including distributed algorithms. All derived algorithms come with formal convergence proofs, demonstrating the framework’s generality, mathematical rigor, and practical applicability.

Technology Category

Search and Optimization: Non-convex OptimizationMachine Learning: OptimizationReasoning under Uncertainty: Stochastic Optimization

Application Category

Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphsEconomics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystemsResponsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

We present a novel methodology for convex optimization algorithm design using ideas from electric RLC circuits. Given an optimization problem, the first stage of the methodology is to design an appropriate electric circuit whose continuous-time dynamics converge to the solution of the optimization problem at hand. Then, the second stage is an automated, computer-assisted discretization of the continuous-time dynamics, yielding a provably convergent discrete-time algorithm. Our methodology recovers many classical (distributed) optimization algorithms and enables users to quickly design and explore a wide range of new algorithms with convergence guarantees.

Problem

• Optimization Algorithms
• Simplification
• Acceleration

Innovation

• RLC Circuit Concept
• Optimization Algorithms
• Convergence Guarantee