quadratic programming feature selection

Embedding decision trees—including ensembles—into optimization problems suffers from low modeling accuracy and poor computational efficiency due to weak linear relaxations in existing mixed-integer programming (MIP) formulations. Method: We propose an ideal MIP formulation based on the union of projection polytopes, explicitly capturing tree logic via binary feature representations and extending to one-dimensional continuous features. Contribution/Results: We prove, for the first time under binary feature encoding, that allowing repeated splits on the same feature eliminates fractional extreme points in the linear relaxation. We further derive the ideal MIP characterization for univariate continuous features. Our formulation substantially tightens the linear relaxation and reduces the number of extreme points in the feasible region. On low-dimensional feature instances, average solution time decreases by an order of magnitude. This advancement significantly improves both the embeddability of tree models into optimization frameworks and their computational scalability.

This paper studies the minimization of separable convex functions over integer feasible sets defined by constraint matrices with small coefficients and bounded primal/dual tree depths. For this class of sparse integer programs, we present the first near-linear-time algorithm for nonlinear separable convex objectives that matches information-theoretic lower bounds. Our method introduces a unified framework integrating scaling techniques, proximity analysis, sensitivity theory, and dynamic data structures. When parameterized by the primal tree depth, the algorithm achieves the optimal time complexity $O(n log |u-l|_infty)$. When parameterized by the dual tree depth, it runs in $O(g n log n log |u-l|_infty)$ time—nearly matching the conjectured optimal bound. These results substantially extend the frontier of efficient solvability for convex integer programming, particularly for structured sparse instances.

This work addresses the issue that quadratic penalty relaxations of binary linear programs often yield spurious or infeasible local minima. To overcome this, we propose a class of QUBO relaxation models satisfying specific structural conditions that guarantee all local minima are feasible and strictly binary. Leveraging these conditions, we derive novel differentiable relaxations for classical combinatorial optimization problems—including open-pit mining, the 0–1 knapsack problem, and the traveling salesman problem—and solve them using gradient-based optimizers such as projected gradient descent and Adam. Experimental results demonstrate that the proposed approach reliably converges to valid binary solutions, thereby establishing clear theoretical guarantees and delineating the applicability boundaries of differentiable optimization as a local solver for combinatorial problems.

To address the low online solving efficiency of Mixed-Integer Programming (MIP), this paper proposes PreMIO: a framework that pretrains lightweight machine learning models on offline data to devise the first data-driven, multi-variable branching strategy with both theoretical provability and interpretability. Leveraging concentration inequalities, the strategy guides hyperplane-based cuts to dynamically partition the feasible region—bridging the long-standing gap between theoretical guarantees and engineering practicality in ML-augmented MIP. PreMIO seamlessly integrates with mainstream MIP solvers without modifying their core algorithms. Evaluated on standard operations research benchmarks (e.g., MIPLIB) and real-world industrial instances, PreMIO reduces average solving time by 32%–57% and significantly decreases the number of explored nodes, demonstrating strong generalization and deployment feasibility.

This paper addresses the trade-off in modeling disjunctive constraints in mixed-integer programming (MIP): the big-M formulation yields weak relaxations, while the convex-hull formulation is computationally expensive. We propose the *P*-split method—a novel reformulation paradigm situated between these extremes—by performing dimensional lifting and piecewise convex-hull construction over additively separable convex functions. Crucially, *P*-split establishes the first hierarchical family of formulations whose relaxation strength monotonically improves with the split parameter *P*, enabling progressive convergence from the big-M relaxation to the convex hull. The method unifies modeling for both convex and nonconvex disjunctive terms, extending beyond classical applicability limits. Evaluated on 344 benchmark instances—including K-means clustering, semi-supervised clustering, P_ball problems, and ReLU neural network optimization—*P*-split achieves node counts comparable to the convex hull while reducing solution time by an order of magnitude, significantly outperforming big-M.

This work addresses polynomial integer programming problems involving high-order variable interactions—a class of optimization challenges significantly more difficult than linear integer programs due to their nonlinear structure. The paper introduces the first solution framework based on hypergraph neural networks, which constructs a high-order-term-aware hypergraph representation to uniformly capture complex dependencies among variables, high-order terms, and constraints. A dual-path hypergraph convolution mechanism is designed to separately aggregate variable–high-order-term and variable–constraint information for predicting high-quality initial solutions, subsequently refined via heuristic search. Experiments demonstrate that the proposed method substantially outperforms existing learning-based approaches and commercial solvers across multiple benchmarks, achieving notable advances in both solution quality and computational efficiency, and is applicable to general polynomial integer programming scenarios.

This work addresses the computational inefficiency in solving mixed-integer convex optimization problems involving binary indicator variables that govern continuous variables. To tackle this challenge, the authors propose the Coordinate Optimality Reconstruction (CORe) framework, which uniquely integrates coordinate-wise optimality conditions into the modeling of indicator variables. By combining closed-form characterizations with disjunctive reformulation techniques, CORe constructs a novel mixed-integer convex programming formulation that effectively exploits exploitable structures embedded in the problem’s sparsity pattern. The approach preserves global optimality while substantially enhancing the performance of branch-and-bound algorithms. Experimental results demonstrate that, across multiple problem classes—including quadratic programs and robust single-index models—CORe significantly accelerates solver convergence compared to conventional big-M formulations.

This work addresses the computational challenges of solving large-scale convex mixed-integer quadratic programs (MIQPs), which arise in applications such as subset portfolio selection and become particularly difficult when the covariance matrix has a high condition number or weight constraints are tight. To tackle this, the authors propose DASH, a novel method that introduces a decreasing active-set hierarchy for dimensionality reduction in MIQP for the first time. DASH leverages active-set analysis to reduce problem dimensionality and integrates seamlessly with commercial solvers like Gurobi to enhance optimization efficiency. Experimental results demonstrate that DASH significantly outperforms Gurobi alone on a range of challenging portfolio instances, with solution quality improvements positively correlated with problem difficulty, thereby accelerating convergence and yielding higher-quality optimal solutions.

This work addresses the challenge of solving mixed binary quadratic programming (MBQP) problems, which are notoriously difficult and for which existing heuristics often fail to produce high-quality feasible solutions within limited time. The authors propose a machine learning–based primal heuristic featuring a novel neural network architecture tailored for MBQP, coupled with an efficient training data collection pipeline. To enhance model generalization, they integrate contrastive loss with weighted cross-entropy loss and introduce a cross-regime transfer inference mechanism. Extensive evaluations on both standard and real-world MBQP benchmarks demonstrate that the proposed method significantly outperforms state-of-the-art heuristics and commercial solvers. Furthermore, it exhibits strong generalization capabilities in practical applications, notably in wind farm layout optimization.