🤖 AI Summary
This work addresses the sample complexity of Sample Average Approximation (SAA) for convex and strongly convex stochastic programming (SP) under standard SP assumptions—without requiring uniform Lipschitz continuity. Methodologically, it integrates convex analysis, stochastic optimization, and functional inequalities to bypass entropy-based arguments. The key contribution is the first tight, metric-entropy-free sample complexity bounds: $O(1/varepsilon^2)$ for convex SP and $O(1/varepsilon)$ for strongly convex SP—improving upon state-of-the-art bounds by a factor of $O(d)$ by eliminating dependence on covering numbers or Rademacher complexity. Theoretically, it reveals that SAA achieves nearly identical sample efficiency as stochastic mirror descent, bridging a long-standing gap in theoretical understanding. Numerical experiments validate the tightness of the bounds and demonstrate that SAA exhibits provably superior practical performance over stochastic mirror descent in non-Lipschitz settings.
📝 Abstract
This paper studies sample average approximation (SAA) in solving convex or strongly convex stochastic programming (SP) problems. In estimating SAA's sample efficiency, the state-of-the-art sample complexity bounds entail metric entropy terms (such as the logarithm of the feasible region's covering number), which often grow polynomially with problem dimensionality. While it has been shown that metric entropy-free complexity rates are attainable under a uniform Lipschitz condition, such an assumption can be overly critical for many important SP problem settings. In response, this paper presents perhaps the first set of metric entropy-free sample complexity bounds for the SAA under standard SP assumptions -- in the absence of the uniform Lipschitz condition. The new results often lead to an $O(d)$-improvement in the complexity rate than the state-of-the-art. From the newly established complexity bounds, an important revelation is that SAA and the canonical stochastic mirror descent (SMD) method, two mainstream solution approaches to SP, entail almost identical rates of sample efficiency, lifting a theoretical discrepancy of SAA from SMD also by the order of $O(d)$. Furthermore, this paper explores non-Lipschitzian scenarios where SAA maintains provable efficacy but the corresponding results for SMD remain mostly unexplored, indicating the potential of SAA's better applicability in some irregular settings. Our numerical experiment results on SAA for solving a simulated SP problem align with our theoretical findings.