Search Strategies for Optimal Classification and Regression Trees

2026-07-30

AI Summary

This work addresses the limited scalability of optimal decision tree construction and the lack of a systematic understanding of existing search strategies. The authors propose a unified algorithmic framework that integrates diverse search strategies within a common modeling paradigm, combining global optimization with pruning techniques to accelerate the solution of optimal classification and regression trees. This framework offers, for the first time, a cohesive perspective enabling direct comparison, hybridization, and principled design of search strategies, supported by an empirical evaluation of 18 distinct approaches. Experimental results demonstrate substantial improvements in anytime performance for classification tasks and achieve over an order-of-magnitude reduction in runtime for regression tasks compared to the current state-of-the-art methods.

Abstract

Optimal decision trees (ODTs) are compact, interpretable machine learning models that globally optimize a given objective, but their scalability remains challenging. While recent work has proposed a variety of search strategies to improve scalability, the precise contribution of each strategy remains unclear. To address this gap, we introduce a general algorithmic framework for ODTs that instantiates previously used search strategies and enables the definition of new ones. This provides a common lens through which to understand and compare different strategies, which we use to empirically investigate the effect of 18 search strategies. Compared to the state of the art, the best strategy in our evaluation achieves significantly better anytime performance for classification, and improves runtime by more than an order of magnitude for regression.

Problem

• Optimal Decision Trees
• Search Strategies
• Scalability
• Classification
• Regression

Innovation

• optimal decision trees
• search strategies
• algorithmic framework
• anytime performance
• scalability