🤖 AI Summary
This paper addresses parameter estimation in statistical models with nonconvex regularizers—such as ℓ₀, SCAD, and MCP—by proposing the first decision diagram–based global optimization framework operating directly in the original variable space. The method constructs a compact convex relaxation via a graph-structured representation and integrates it into a spatial branch-and-bound scheme, eliminating the need for auxiliary binary variables or artificial variable bounds. It unifies treatment of generalized ℓₚ norms (p ∈ [0, ∞)) and intricate nonconvex penalty functions. Compared to existing approaches, it achieves superior generalizability and tighter relaxations while guaranteeing finite-time convergence to the global optimum under mild assumptions. Empirically, on sparse linear regression benchmarks, it is the first method to successfully solve SCAD- and MCP-regularized instances that are intractable for conventional global optimizers—demonstrating feasibility, numerical accuracy, and theoretical soundness.
📝 Abstract
Optimization problems with norm-bounding constraints arise in a variety of applications, including portfolio optimization, machine learning, and feature selection. A common approach to these problems involves relaxing the norm constraint via Lagrangian relaxation, transforming it into a regularization term in the objective function. A particularly challenging class includes the zero-norm function, which promotes sparsity in statistical parameter estimation. Most existing exact methods for solving these problems introduce binary variables and artificial bounds to reformulate them as higher-dimensional mixed-integer programs, solvable by standard solvers. Other exact approaches exploit specific structural properties of the objective, making them difficult to generalize across different problem types. Alternative methods employ nonconvex penalties with favorable statistical characteristics, but these are typically addressed using heuristic or local optimization techniques due to their structural complexity. In this paper, we propose a novel graph-based method to globally solve optimization problems involving generalized norm-bounding constraints. Our approach encompasses standard $ell_p$-norms for $p in [0, infty)$ and nonconvex penalties such as SCAD and MCP. We leverage decision diagrams to construct strong convex relaxations directly in the original variable space, eliminating the need for auxiliary variables or artificial bounds. Integrated into a spatial branch-and-cut framework, our method guarantees convergence to the global optimum. We demonstrate its effectiveness through preliminary computational experiments on benchmark sparse linear regression problems involving complex nonconvex penalties, which are not tractable using existing global optimization techniques.