Lower Complexity Bounds for Nonconvex-Strongly-Convex Bilevel Optimization with First-Order Oracles

2025-11-24

AI Summary

This work establishes tight complexity lower bounds for nonconvex-strongly-convex bilevel optimization under standard first-order deterministic and stochastic oracle models. To bridge the significant gap between existing upper and lower bounds, the authors construct the first “hard instance” specifically tailored to the bilevel structure. Their analysis integrates zero-respecting algorithm frameworks, smoothing techniques, and tools from complexity theory. They derive the first tight lower bounds: Ω(κ^{3/2}ε^{-2}) for the deterministic setting and Ω(κ^{5/2}ε^{-4}) for the stochastic setting, where κ denotes the condition number of the lower-level problem. These bounds strictly improve upon known lower bounds for both single-level nonconvex optimization and nonconvex-strongly-convex minimax problems. The results fundamentally characterize the intrinsic computational hardness induced by the bilevel nesting structure and provide a theoretical benchmark for the optimal complexity of first-order methods in bilevel optimization.

Technology Category

• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Security and Privacy: Large-scale security measurements

Abstract

Although upper bound guarantees for bilevel optimization have been widely studied, progress on lower bounds has been limited due to the complexity of the bilevel structure. In this work, we focus on the smooth nonconvex-strongly-convex setting and develop new hard instances that yield nontrivial lower bounds under deterministic and stochastic first-order oracle models. In the deterministic case, we prove that any first-order zero-respecting algorithm requires at least $Ω(κ^{3/2}ε^{-2})$ oracle calls to find an $ε$-accurate stationary point, improving the optimal lower bounds known for single-level nonconvex optimization and for nonconvex-strongly-convex min-max problems. In the stochastic case, we show that at least $Ω(κ^{5/2}ε^{-4})$ stochastic oracle calls are necessary, again strengthening the best known bounds in related settings. Our results expose substantial gaps between current upper and lower bounds for bilevel optimization and suggest that even simplified regimes, such as those with quadratic lower-level objectives, warrant further investigation toward understanding the optimal complexity of bilevel optimization under standard first-order oracles.

Problem

• Establishes lower complexity bounds for nonconvex-strongly-convex bilevel optimization problems
• Develops hard instances for deterministic and stochastic first-order oracle models
• Reveals substantial gaps between current upper and lower complexity bounds

Innovation

• Developed hard instances for lower bounds
• Proved deterministic first-order oracle complexity
• Established stochastic first-order oracle complexity