A Generic Branch-and-Bound Algorithm for $ell_0$-Penalized Problems with Supplementary Material

2025-06-04

AI Summary

This paper addresses the $ell_0$-regularized optimization problem under general differentiable loss functions. We propose the first branch-and-bound (B&B) framework applicable to arbitrary losses—not restricted to quadratic—overcoming limitations of prior methods relying on “Big-M” or $ell_2$ relaxations. Our key methodological contribution is a unified, flexible relaxation theory encompassing multiple relaxation strategies, enabling closed-form expressions for all critical B&B quantities: dual bounds, branching rules, and node pruning conditions. Based on this theory, we develop El0ps, an open-source solver supporting user-defined losses and regularizers for plug-and-play $ell_0$ modeling. Experiments demonstrate state-of-the-art performance on classical benchmarks; notably, El0ps is the first to provably solve large-scale and non-quadratic $ell_0$ problems, substantially expanding the computational tractability frontier of $ell_0$ optimization.

Technology Category

• Search and Optimization: Non-convex Optimization
• Constraint Satisfaction and Optimization: Solvers and Tools
• Machine Learning: Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We present a generic Branch-and-Bound procedure designed to solve L0-penalized optimization problems. Existing approaches primarily focus on quadratic losses and construct relaxations using"Big-M"constraints and/or L2-norm penalties. In contrast, our method accommodates a broader class of loss functions and allows greater flexibility in relaxation design through a general penalty term, encompassing existing techniques as special cases. We establish theoretical results ensuring that all key quantities required for the Branch-and-Bound implementation admit closed-form expressions under the general blanket assumptions considered in our work. Leveraging this framework, we introduce El0ps, an open-source Python solver with a plug-and-play workflow that enables user-defined losses and penalties in L0-penalized problems. Through extensive numerical experiments, we demonstrate that El0ps achieves state-of-the-art performance on classical instances and extends computational feasibility to previously intractable ones.

Problem

• Solves L0-penalized optimization problems generically
• Accommodates diverse loss functions and flexible relaxations
• Provides open-source solver for previously intractable cases

Innovation

• Generic Branch-and-Bound for L0-penalized problems
• Flexible relaxation design with general penalty term
• Open-source Python solver El0ps for user-defined losses