Generalized Optimal Classification Trees: A Mixed-Integer Programming Approach

2026-02-02

AI Summary

This work addresses the challenge of directly optimizing nonlinear metrics—such as the F1 score—in global decision tree optimization, particularly under class imbalance. The authors propose a mixed-integer programming (MIP)-based framework for learning interpretable classification trees that, for the first time, enables direct optimization of such nonlinear objectives. To enhance computational efficiency and scalability, they introduce problem-specific branch-and-cut strategies, instance reduction techniques, and warm-start heuristics. Extensive experiments on 50 benchmark datasets demonstrate that the proposed method achieves superior predictive performance while significantly reducing solution times compared to state-of-the-art optimal classification tree approaches.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Optimization
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Global optimization of decision trees is a long-standing challenge in combinatorial optimization, yet such models play an important role in interpretable machine learning. Although the problem has been investigated for several decades, only recent advances in discrete optimization have enabled practical algorithms for solving optimal classification tree problems on real-world datasets. Mixed-integer programming (MIP) offers a high degree of modeling flexibility, and we therefore propose a MIP-based framework for learning optimal classification trees under nonlinear performance metrics, such as the F1-score, that explicitly addresses class imbalance. To improve scalability, we develop problem-specific acceleration techniques, including a tailored branch-and-cut algorithm, an instance-reduction scheme, and warm-start strategies. We evaluate the proposed approach on 50 benchmark datasets. The computational results show that the framework can efficiently optimize nonlinear metrics while achieving strong predictive performance and reduced solution times compared with existing methods.

Problem

• optimal classification trees
• class imbalance
• nonlinear performance metrics
• global optimization
• interpretable machine learning

Innovation

• Mixed-Integer Programming
• Optimal Classification Trees
• F1-score Optimization
• Class Imbalance
• Branch-and-Cut Algorithm