RCProb: Probabilistic Rule Extraction for Efficient Simplification of Tree Ensembles

2026-04-28

AI Summary

This work addresses the declining interpretability of tree ensemble models as they scale up, compounded by the high computational cost of existing rule extraction methods on large datasets. To overcome these limitations, the authors propose an efficient rule extraction approach grounded in probabilistic modeling. By incorporating a Dirichlet prior and Beta smoothing into a naive Bayes estimation framework, the method eliminates the need for repeated scans of the training data, substantially reducing the computational overhead of confidence estimation. Evaluated across 33 benchmark datasets, the proposed technique achieves an average speedup of approximately 22×, yields more compact rule sets, and maintains predictive accuracy comparable to that of the original ensemble model.

Abstract

Tree ensembles are widely used in industrial machine learning due to their strong predictive performance and efficient training procedures. However, as the number of trees in an ensemble grows, the resulting models become increasingly difficult for humans to interpret. To address this limitation, explainable artificial intelligence (XAI) studies methods that generate interpretable models capable of explaining complex predictors. One approach consists of extracting decision rules from tree ensembles while attempting to preserve the predictive performance of the original model. In previous work, we introduced RuleCOSI+, a greedy heuristic algorithm for extracting compact rule-based models from tree ensembles. Although RuleCOSI+ produces accurate and interpretable rule sets, it relies on repeated empirical frequency counting over the training data to estimate rule confidence, which becomes computationally expensive for large datasets. In this paper, we propose RCProb, a probabilistic reformulation of RuleCOSI+ designed to reduce the computational cost of rule extraction. RCProb estimates rule statistics using Dirichlet-smoothed class priors and Beta-smoothed condition likelihoods combined through a Naive Bayes formulation, avoiding repeated dataset scans. Experiments on 33 benchmark datasets show that RCProb maintains competitive predictive performance while reducing runtime by approximately $22\times$ compared with RuleCOSI+, while producing more compact rule sets on average.

Problem

• tree ensembles
• rule extraction
• explainable AI
• computational efficiency
• interpretability

Innovation

• probabilistic rule extraction
• tree ensembles
• explainable AI
• Dirichlet smoothing
• Naive Bayes