analytic optimization

Existing approaches for identifying structured constraints—such as one-hot and special-ordered set (SOS) constraints—in mathematical optimization modeling suffer from low efficiency and poor robustness, especially when handling complex, nested algebraic expressions. Method: This paper proposes a symbolic-level constraint identification method based on e-graphs, pioneering the integration of the egg e-graph framework into industrial-grade optimization modeling systems. We design an algebraic congruence–driven heuristic rewriting system and develop egg_recursive, an open-source library supporting recursive abstract syntax tree (AST) representations to simplify maintenance of complex S-expressions. Contribution/Results: The method is implemented and deployed in JijModeling, significantly improving constraint identification accuracy and generalization across diverse modeling patterns. Benchmark evaluations demonstrate a 3.2× speedup in preprocessing time. The approach has been successfully applied to real-world quantum and hybrid optimization tasks, validating its engineering practicality, scalability, and production readiness.

This work addresses the lack of machine-verifiable formalizations of line search methods in nonlinear optimization, which has hindered algorithmic reliability. Within the Lean 4 theorem prover, it presents the first systematic formalization of several classical line search criteria—including Armijo, Goldstein, Wolfe, and their nonmonotone variants—alongside rigorous definitions of gradient descent, descent directions, and backtracking step-size selection. The study fully verifies the Zoutendijk convergence theorem within this framework, thereby establishing a comprehensive formal foundation for line search theory. This contribution significantly enhances the verifiability and trustworthiness of nonlinear optimization algorithms through mechanized mathematical reasoning.

This work addresses discrete-time nonlinear optimal control problems by unifying classical algorithms—including gradient descent, Gauss–Newton, Newton’s method, and differential dynamic programming (DDP)—within a differentiable programming framework. Methodologically, it introduces the first modular, end-to-end differentiable algorithm template library built upon linear/quadratic approximations (e.g., LQR), enabled by automatic differentiation. Theoretically, it provides a unified derivation of computational complexity and sufficient optimality conditions across all methods. Practically, it incorporates adaptive line search and regularization strategies, and validates efficacy on benchmark tasks such as autonomous racing with a bicycle model. All implementations are open-sourced, demonstrating both efficient gradient propagation and strong generalization across diverse control problems.

There exists a significant gap between the theoretical convergence guarantees of deep learning optimization algorithms and their empirical performance, largely due to commonly adopted assumptions—such as Hessian boundedness—that lack empirical validation. Method: We introduce the first trajectory-aware measurement framework tightly aligned with key theoretical quantities, systematically evaluating the validity of mainstream assumptions across diverse architectures and datasets using large-scale training runs. Our framework quantifies dynamic properties—including gradient norms, Hessian spectral characteristics, and loss curvature—along optimization trajectories. Contribution/Results: We find that all examined theoretical assumptions fail to reliably predict actual convergence behavior and exhibit no robust correlation with optimization performance. This work uncovers a fundamental misalignment between theoretical modeling and practice, establishing the first reproducible benchmark for empirically calibrating and reconstructing optimization theory.

This work addresses the convergence guarantees of stochastic line search optimization for over-parameterized models under interpolation conditions. We establish a necessary and sufficient condition on the search direction—applicable to a broad class of methods—that ensures finite termination and bounded backtracking steps, and rigorously prove linear convergence under the Polyak–Łojasiewicz (PL) assumption. The condition unifies major first-order strategies—including momentum, conjugate gradient, and adaptive preconditioning—providing a verifiable theoretical foundation for their principled integration with stochastic line search. Our analysis fills a critical gap in the convergence theory of stochastic line search methods and significantly extends both the applicability and reliability of efficient first-order optimization in interpolation learning regimes.

This work addresses the challenges posed by higher-order functions in mathematical optimization modeling, which often lead to unnatural LaTeX output and inefficient constraint verification. To overcome these issues, the authors propose an egglog-based optimization approach that performs desugaring reconstruction on the λ-calculus intermediate representation of JijModeling 2, thereby recovering comprehension-like syntactic structures. By integrating Henkin-style constants with Datalog-inspired rules, the method enables declarative, multi-step constraint checking. A custom cost model and equality saturation techniques are introduced to enhance performance significantly: the generated LaTeX aligns more closely with conventional mathematical notation, and complex constraint validation—previously requiring minutes or failing to terminate—now completes within seconds.

This work proposes a trajectory-restricted framework for linear convergence analysis that overcomes the conservatism of traditional first-order methods, whose guarantees often rely on global geometric conditions and worst-case constants. Instead of imposing regularity assumptions globally, our approach requires only local geometric properties—such as restricted Polyak–Łojasiewicz inequalities, error bounds, and quadratic growth—on the subset of the space actually traversed by the algorithm. We establish explicit relationships among the associated constants and show that, for piecewise polyhedral composite problems, once iterates enter a well-conditioned active manifold, convergence is governed by the restricted Hoffman constant of that manifold, yielding an improved effective condition number and faster local convergence. The results demonstrate that linear convergence fundamentally depends on the local geometry encountered along the algorithmic trajectory, rather than on global worst-case scenarios.

This work addresses the long-standing perception that zeroth-order (ZO) optimization algorithms inherently converge more slowly than first-order (FO) methods due to dimensionality dependence. By interpreting ZO algorithms through the lens of dynamical systems, the paper models them as perturbed averaged versions of FO algorithms and introduces, for the first time, the input-to-state stability (ISS) theoretical framework to analyze their convergence behavior. Under suitable conditions, the authors rigorously establish that ZO methods can eliminate the dimensionality gap in expectation, converging at the same rate as FO methods to an arbitrarily small neighborhood of the FO fixed point. These theoretical findings are corroborated by numerical experiments, highlighting the underappreciated convergence potential of ZO optimization algorithms.

This work proposes a novel paradigm termed “Optimized Natural Physics” to investigate whether optimization algorithms adhere to natural laws of motion induced by the objective function. By establishing equivalence between optimal control problems and generalized KKT conditions, the authors construct a natural vector field governed by non-Newtonian dynamics. Leveraging Pontryagin’s minimum principle, Hamilton–Jacobi inequalities, and energy dissipation mechanisms, they design control strategies possessing inverse optimality. This framework not only unifies the interpretation of diverse existing optimization algorithms but also enables the systematic derivation of new ones. The approach demonstrates that global optimization can be achieved through deliberate modulation of jumps and dissipation, thereby providing a physically intuitive and mathematically unified foundation for optimization theory.

Current large language models lack mechanisms to verify the structural soundness and solution validity of automatically generated mathematical optimization models, which limits their modeling accuracy. This work proposes Opt-Verifier, a novel framework that introduces a dual-loop verification mechanism by jointly assessing generated models along two dimensions: structural consistency and solution validity. By enforcing rigorous cross-checks between model formulation and feasible solutions, Opt-Verifier significantly enhances the logical coherence and mathematical correctness of the generated models. Empirical evaluations on standard benchmarks demonstrate that this approach improves modeling accuracy by over 20%, establishing a new paradigm for reliable automated optimization modeling.