continuous relaxation

In decision-focused learning (DFL), the gradients of predicted parameters—derived from optimization problems such as linear programming—are almost everywhere zero, causing gradient vanishing when directly optimizing decision regret. Existing smoothing or surrogate loss approaches fail to resolve this fundamental issue. This paper proposes a surrogate-loss-driven end-to-end framework: even when employing differentiable optimization layers (e.g., DYS-Net), we explicitly minimize a carefully designed surrogate loss to ensure consistent gradient flow and alignment with decision quality. Theoretically, we prove that smoothing cannot eliminate the inherent zero-measure set of non-differentiability, whereas our surrogate loss circumvents this bottleneck entirely. Experiments demonstrate that our method achieves state-of-the-art or superior decision regret across benchmarks, while accelerating training significantly—delivering both higher accuracy and computational efficiency.

Deep neural networks struggle to directly optimize non-differentiable objectives (e.g., IoU, BLEU, reward signals) due to the absence of well-defined gradients. Method: This paper reframes deep reinforcement learning (DRL) as a generalized extension of supervised learning, centering on gradient-based policy optimization frameworks (e.g., PPO) rather than tabular RL paradigms. It systematically integrates loss proxy modeling, policy gradient derivation, and human feedback alignment techniques to bridge the gap from single-step non-differentiable optimization to multi-step sequential decision-making. Contribution/Results: The work establishes, for the first time, a conceptual continuity between supervised learning and DRL—clarifying theoretical foundations, algorithmic boundaries, and practical implementation logic. This significantly lowers the entry barrier to DRL, enabling researchers with only supervised learning background to rigorously understand, adapt, and deploy state-of-the-art DRL methods across diverse application domains.

Policy gradient methods suffer from inherent high variance and low sample efficiency. This paper proposes SoftTreeMax—a differentiable generalization of the softmax operator that integrates multi-step planning by jointly modeling future-state logits and discounted or exponentiated cumulative rewards. We theoretically characterize, for the first time, how tree expansion depth governs gradient variance, proving its exponential decay with planning depth. We further design a differentiable tree-structured policy enabling gradient reuse across all leaf nodes. Leveraging GPU-parallelized simulation, differentiable tree architecture, and unified gradient backpropagation, SoftTreeMax reduces gradient variance by three orders of magnitude on Atari benchmarks, achieves five times the performance of distributed PPO, and shortens training time. Empirical analysis confirms a strong positive correlation between reduced gradient variance and improved return.

This work addresses the family of parametric optimization problems and proposes the first unified, data-driven framework for analyzing the generalization performance of both classical and learned optimizers. Methodologically: (1) it introduces PAC-Bayes theory to the analysis of learned optimizers, deriving verifiable, high-probability generalization upper bounds; (2) it establishes performance bounds for classical optimizers based on empirical convergence rates; and (3) it pioneers a learning paradigm that directly minimizes the PAC-Bayes bound during training. Evaluated on signal processing, control, and meta-learning tasks, the derived bounds are significantly tighter than conventional worst-case guarantees. Moreover, the theoretical generalization guarantees for learned optimizers consistently exceed the empirical performance of their non-learned baselines—thereby unifying theoretical rigor with practical efficacy.

This paper investigates the computational complexity of finding (δ,ε)-stable points in stochastic nonsmooth nonconvex optimization. To overcome the bottleneck of the prior best-known complexity O(ε⁻⁴δ⁻¹), we establish, for the first time, a rigorous reduction framework from nonsmooth nonconvex optimization to online learning, reformulating the problem as an optimistic online learning task and integrating stochastic subgradient methods with complexity lower-bound analysis. Our contributions are: (1) achieving the optimal stochastic gradient query complexity O(ε⁻³δ⁻¹); (2) deriving a tight lower bound that certifies its theoretical optimality; (3) naturally extending the analysis to the second-order smooth setting, yielding a new complexity bound O(ε⁻¹·⁵δ⁻⁰·⁵); and (4) unifying and recovering all known optimal or state-of-the-art results for smooth and higher-order smooth settings—thereby establishing a paradigm-level unification.

This work addresses the inefficiency of global optimization when standard neural network surrogates are embedded into mixed-integer linear programs (MILPs), a challenge stemming from the lack of control over their structural properties. The authors propose a novel differentiable regularizer that, for the first time, approximates the full gradient of the LP relaxation gap with respect to network parameters, enabling direct optimization of key structural attributes such as big-M constants, the number of unstable neurons, and the LP relaxation gap itself. Built upon ReLU networks and MILP formulations, the method leverages gradients from LP dual variables and requires no custom automatic differentiation. Experiments demonstrate up to four orders of magnitude reduction in MILP solve time on nonconvex benchmark functions and two-stage stochastic programming problems, all while preserving predictive accuracy.

This work addresses the issue of undefined and unstable gradients in deterministic policy gradient methods under sparse or discrete reward settings, where the Q-function is non-differentiable with respect to actions. To overcome this limitation, the paper proposes Soft Deterministic Policy Gradient (Soft-DPG), which introduces Gaussian smoothing into the deterministic policy gradient framework for the first time. By constructing a smoothed Bellman equation and redefining the action-value function, Soft-DPG circumvents the explicit reliance on the gradient of the Q-function with respect to actions. Theoretical analysis demonstrates that the proposed method ensures well-defined policy gradients even when the Q-function is non-smooth. Empirical results show that Soft-DPG achieves competitive performance in standard continuous control tasks with dense rewards and significantly outperforms DDPG in environments with sparse or discrete rewards.

This work addresses the computational challenges posed by traditional discrete model selection criteria—such as AIC and BIC—whose objective functions are discontinuous due to their dependence on the number of parameters, rendering them difficult to optimize. To overcome this limitation, the authors propose a provably convergent sequence of smooth surrogate functions that reformulate these criteria as continuous optimization problems. By integrating this approach with explicit over-parameterized modeling, the method enables joint optimization of model selection and parameter inference. This framework circumvents the computational bottlenecks inherent in conventional discrete optimization strategies, significantly reducing computational complexity while achieving superior performance in clustering tasks.

This work addresses the high computational cost and optimization instability of existing Monte Carlo sampling–based variational inference methods for learning neural set functions under weak supervision. The authors propose a sampling-free, continuous relaxation learning framework that reinterprets the evidence lower bound (ELBO) as a continuous relaxation of set functions and introduces a learnable surrogate objective to yield stable and efficient gradients. The approach enjoys approximation guarantees under submodular maximization and reveals a theoretical connection to variational free energy. Experimental results demonstrate that the method significantly outperforms current baselines across multiple real-world tasks, achieving faster convergence and substantially reduced computational overhead.

This work addresses the challenge in bilevel optimization where non-isolated minima manifolds in the lower-level problem render the upper-level objective nondifferentiable. To overcome this, the authors propose a “select-then-differentiate” framework that, under a local Polyak–Łojasiewicz condition, introduces a unique optimistic selection to ensure hypergradient differentiability and enables explicit hypergradient computation via the pseudoinverse. Theoretically, global uniqueness of the lower-level solution is unnecessary; local smoothness of the upper-level objective is guaranteed as long as the selected solution is nondegenerate on the manifold. The proposed HG-MS method converges to stationary points of the optimistic objective, with complexity governed by the intrinsic dimension of the manifold. Empirically, it significantly outperforms existing approaches on LLM source reweighting tasks, achieving state-of-the-art results on GSM8K and MATH benchmarks and leading performance on MT-Bench.