Problem
Research questions and friction points this paper is trying to address.
Global optimizers for nonconvex problems
Adaptive tuning of stochastic terms
Algebraic convergence in probability and parameters
An Algebraically Converging Stochastic Gradient Descent Algorithm for Global Optimization
🤖 AI Summary
Addressing the challenge of global convergence in nonconvex optimization, this paper proposes a state-dependent stochastic perturbation gradient descent algorithm, where the noise intensity is adaptively modulated by the current objective function value—akin to a “state-dependent temperature”—thereby overcoming the limitation of predefined time-varying temperature schedules in classical simulated annealing. Theoretically, we establish, for the first time within a discrete-time algorithmic framework, rigorous global convergence in both probability and parameter space, with an algebraic convergence rate. Technically, the method integrates stochastic gradient updates, state-driven noise modeling, and discrete-time Markov chain analysis. Numerical experiments on canonical hard nonconvex problems demonstrate that the algorithm achieves superior efficiency and robustness compared to fixed-noise baselines and classical annealing-type methods.
📝 Abstract
We propose a new gradient descent algorithm with added stochastic terms for finding the global optimizers of nonconvex optimization problems. A key component in the algorithm is the adaptive tuning of the randomness based on the value of the objective function. In the language of simulated annealing, the temperature is state-dependent. With this, we prove the global convergence of the algorithm with an algebraic rate both in probability and in the parameter space. This is a significant improvement over the classical rate from using a more straightforward control of the noise term. The convergence proof is based on the actual discrete setup of the algorithm, not just its continuous limit as often done in the literature. We also present several numerical examples to demonstrate the efficiency and robustness of the algorithm for reasonably complex objective functions.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Stochastic gradient descent algorithm
Adaptive tuning of randomness
State-dependent temperature control
🔎 Similar Papers
No similar papers found.
