🤖 AI Summary
This work addresses the poor sample efficiency of Monte Carlo estimators in derivative-free stochastic nonconvex optimization by proposing a hierarchical adaptive sampling strategy within a trust-region framework. The method dynamically controls the Monte Carlo estimation error, ensuring it remains below a stationarity threshold derived from the trust-region radius, thereby achieving high accuracy while substantially reducing sample complexity. Theoretical analysis establishes an improved sample complexity bound for the proposed algorithm, and numerical experiments further demonstrate its computational efficiency and optimization accuracy in high-dimensional settings.
📝 Abstract
There is emerging evidence that trust-region (TR) algorithms are very effective at solving derivative-free nonconvex stochastic optimization problems in which the objective function is a Monte Carlo (MC) estimate. A recent strand of methodologies adaptively adjusts the sample size of the MC estimates by keeping the estimation error below a measure of stationarity induced from the TR radius. In this work we explore stratified adaptive sampling strategies to equip the TR framework with accurate estimates of the objective function, thus optimizing the required number of MC samples to reach a given ε-accuracy of the solution. We prove a reduced sample complexity, confirm a superior efficiency via numerical tests and applications, and explore inexpensive implementations in high dimension.