ArborEnum: Decision Tree Rashomon Sets over Continuous Features

2026-08-04

AI Summary

Existing methods for enumerating the Rashomon set of decision trees rely on data binarization, which struggles to accurately capture model diversity under continuous features and incurs substantial computational costs. This work proposes the first exact enumeration algorithm that natively supports continuous features by leveraging their inherent ordering to avoid prespecified discretization. The approach introduces a threshold-relaxation approximation strategy and an anytime optimization framework, progressively refining candidate split points to converge toward the true Rashomon set. Compared to existing techniques, the proposed method achieves speedups of several orders of magnitude; its approximate variant significantly accelerates computation while maintaining near-perfect recall, effectively mitigating the omission of important models and features caused by coarse-grained binarization.

Abstract

The Rashomon effect describes the phenomenon that many models can achieve nearly equivalent performance on the same learning task, with significant ramifications for robustness, feature importance, and customizability. These use cases motivate the computation of Rashomon sets: the set of all models whose regularized loss is near-optimal. Decision trees are one of the few model classes for which Rashomon sets can be fully enumerated, but this computation has always been conditional on a binarization of the original data, either restricting which splits each tree is allowed to make or substantially increasing the complexity of an already difficult combinatorial problem. We introduce the first algorithm that exactly enumerates decision-tree Rashomon sets while exploiting the ordered structure of continuous features. We further develop a relaxation for approximate enumeration and an anytime algorithm that progressively refines the set of candidate thresholds, producing increasingly detailed approximations that converge to the continuous-feature Rashomon set. Experiments show that coarse binarization can miss many trees, important features, and predictive multiplicity; our algorithms achieve orders-of-magnitude speedups over existing enumeration methods, with approximations providing further speedups while maintaining near-perfect recall.

Problem

• Rashomon sets
• decision trees
• continuous features
• enumeration
• predictive multiplicity

Innovation

• Rashomon sets
• decision trees
• continuous features
• exact enumeration
• anytime algorithm