solve variational problems

This paper addresses black-box binary optimization—where the objective function is unknown, unlabeled, and non-differentiable. We propose an unsupervised solution framework that requires neither policy evaluation nor historical solved instances. Our core method constructs an estimator for the optimal subproblem objective value, using the expected deviation from optimality conditions as an unsupervised loss function; this estimator is trained via inequality-constrained function approximation, guiding global search without explicitly solving subproblems. Unlike conventional approaches relying on policy-value estimation or labeled data, our framework leverages in-distribution statistical properties to approximate optimal solutions. Experiments demonstrate its effectiveness and robustness under fully unsupervised settings with unknown objectives.

This study investigates the statistical properties of Lagrange multipliers in constrained maximum likelihood estimation and least squares problems, along with their implications for numerical optimization. Leveraging large-sample theory, it establishes that under correctly specified models, Lagrange multipliers converge in probability to zero as the sample size grows, a result extended to high-dimensional settings such as deep learning. Building on this asymptotic behavior, the work provides the first statistical justification for initializing Lagrange multipliers at zero and integrates this insight into constrained optimization algorithms, including augmented Lagrangian methods and sequential quadratic programming. Numerical experiments demonstrate that this initialization strategy substantially enhances algorithmic stability and convergence efficiency in applications such as constrained regression and dynamic discrete choice models.

This work addresses discrete-time nonlinear optimal control problems by unifying classical algorithms—including gradient descent, Gauss–Newton, Newton’s method, and differential dynamic programming (DDP)—within a differentiable programming framework. Methodologically, it introduces the first modular, end-to-end differentiable algorithm template library built upon linear/quadratic approximations (e.g., LQR), enabled by automatic differentiation. Theoretically, it provides a unified derivation of computational complexity and sufficient optimality conditions across all methods. Practically, it incorporates adaptive line search and regularization strategies, and validates efficacy on benchmark tasks such as autonomous racing with a bicycle model. All implementations are open-sourced, demonstrating both efficient gradient propagation and strong generalization across diverse control problems.

This work addresses the autoformulation problem—automatically translating natural-language problem descriptions into solvable mathematical optimization models. We propose the first LLM-driven Monte Carlo Tree Search (MCTS) framework for this task, enabling dynamic hypothesis generation and formal correctness evaluation. Our method integrates hierarchical optimization modeling representations, LLM-based semantic understanding, and MCTS-based search strategies. A key innovation is an equivalence-aware pruning mechanism that reduces search overhead by over 40%. Empirically, our approach achieves state-of-the-art performance on LP/MIP benchmarks, outperforming all existing baselines. LLM-assisted verification accelerates correctness assessment significantly. Moreover, this work formally defines the autoformulation task for the first time, establishing a scalable, automated paradigm to lower the barrier to optimization modeling and empower domain experts.

This work addresses the convergence guarantees of stochastic line search optimization for over-parameterized models under interpolation conditions. We establish a necessary and sufficient condition on the search direction—applicable to a broad class of methods—that ensures finite termination and bounded backtracking steps, and rigorously prove linear convergence under the Polyak–Łojasiewicz (PL) assumption. The condition unifies major first-order strategies—including momentum, conjugate gradient, and adaptive preconditioning—providing a verifiable theoretical foundation for their principled integration with stochastic line search. Our analysis fills a critical gap in the convergence theory of stochastic line search methods and significantly extends both the applicability and reliability of efficient first-order optimization in interpolation learning regimes.

This work proposes a differentiable programming–based framework for learning adaptive optimization algorithms to address the slow convergence and high per-iteration cost of traditional first-order methods in large-scale optimization. By embedding Fenchel–Rockafellar duality theory into automatic differentiation systems, the framework enables end-to-end training and adaptive refinement of duality-driven iterative schemes such as ADMM and PDHG. Implemented uniformly across major deep learning frameworks—including PyTorch, TensorFlow, and JAX—the approach significantly improves both computational efficiency and solution quality on a range of tasks, including linear programming, optimal power flow (OPF), Laplacian regularization, and neural network verification.

This work addresses the challenges posed by higher-order functions in mathematical optimization modeling, which often lead to unnatural LaTeX output and inefficient constraint verification. To overcome these issues, the authors propose an egglog-based optimization approach that performs desugaring reconstruction on the λ-calculus intermediate representation of JijModeling 2, thereby recovering comprehension-like syntactic structures. By integrating Henkin-style constants with Datalog-inspired rules, the method enables declarative, multi-step constraint checking. A custom cost model and equality saturation techniques are introduced to enhance performance significantly: the generated LaTeX aligns more closely with conventional mathematical notation, and complex constraint validation—previously requiring minutes or failing to terminate—now completes within seconds.

This work addresses the limitations of black-box optimization in structural design, which often yields suboptimal or physically implausible solutions due to its neglect of problem modeling and domain knowledge. Focusing on the topology optimization of laminated composite structures, the study proposes an explicit decoupling of topological and fiber orientation design variables, combined with a physics-informed sequential optimization strategy. This approach departs from conventional context-agnostic black-box paradigms by leveraging domain-specific insights. Compared to concurrent optimization of all variables, the proposed sequential method significantly improves compliance minimization under volume constraints, yielding superior and physically interpretable designs. The results underscore the critical role of integrating domain knowledge into the optimization process to enhance both performance and solution plausibility.

This work investigates whether convex-concave minimax optimization can be solved directly—bypassing the standard variational inequality (VI) reformulation—to achieve faster convergence. Focusing on the canonical setting of unconstrained quadratic objectives, it establishes, for the first time, that the optimal first-order convergence rate for minimax problems strictly outperforms that of their VI counterparts. The key insight is the exploitation of inherent asymmetry between primal and dual variables—a structural property obscured in the VI formulation and responsible for its suboptimal rate. Using extremal polynomial techniques grounded in Green’s functions and conformal mapping, the authors precisely characterize the asymptotic optimal convergence rates for both formulations. Under the standard first-order oracle model, minimax optimization admits a strictly superior asymptotic convergence rate. This result provides new theoretical foundations and a structured pathway for algorithmic acceleration, challenging the conventional VI-based design paradigm.

This work proposes a unified framework based on stochastic control and mean-field theory for solving global optimization problems in both Euclidean space and the Wasserstein space of probability measures. By introducing a regularized stochastic control problem and leveraging dynamic programming, the Cole–Hopf transformation, and the Feynman–Kac formula, the authors derive a computable approximation to the original problem. A system of N interacting particles is then employed to numerically solve the associated measure-valued dynamics. Theoretical analysis establishes that as the regularization parameter vanishes and the number of particles tends to infinity, the value function of the control problem converges to the global minimum of the original optimization problem. Numerical experiments confirm the method’s efficacy and validate the predicted theoretical convergence rates. This study represents the first integration of stochastic control with mean-field theory, offering a novel paradigm and rigorous theoretical guarantees for global optimization across both spaces.