Complexity Bounds and Approaches to Learning Projected Gradient Descent Solver Iterates

2026-07-24

AI Summary

This work addresses the challenge of efficiently training generative models to produce high-quality initial solutions under data scarcity. The authors propose a k-neighborhood data collection strategy that enriches the training set by reusing intermediate iterates from a projected gradient descent solver, thereby improving data efficiency without incurring additional solve overhead. For the first time, they derive a generalization error bound based on Rademacher complexity for learning from solver trajectories, revealing how the k-neighborhood structure influences generalization performance. The approach bridges data-driven dynamic adaptive systems (DDDAS) with the emerging global search method GLENS. Theoretical analysis and experiments on box-constrained quadratic programming problems demonstrate that the proposed method significantly enhances both data utilization and optimization loop efficiency.

Abstract

Data scarcity poses a fundamental challenge in training generative models to produce initial guesses for parametric optimization problems that are otherwise numerically expensive to solve. We therefore study a $k$-neighborhood data collection strategy that augments datasets of converged solutions with intermediate solver iterates, increasing the amount of training data without additional solver runs. To understand the benefits of this approach, we derive a generalization bound based on Rademacher complexity that reveals the role of the $k$-neighborhoods and related parameters. To achieve this result, we focus on one-sided box-constrained quadratic programs solved by projected gradient descent. We illustrate the behavior of this solver on two examples. The approach proposed in this paper enables a more capable DDDAS paradigm by improving the efficiency of the data-model-optimization loop. We finish by discussing two views of learning solver-iterate data and connect our analysis with GLENS, a new data-efficient global search method.

Problem

• data scarcity
• generative models
• initial guesses
• parametric optimization
• solver iterates

Innovation

• k-neighborhood data augmentation
• Rademacher complexity
• projected gradient descent iterates
• generalization bound
• data-efficient optimization