CLARITree: Cholesky and Lookahead Accelerations for Regression with Interpretable Piecewise Linear Trees

2026-06-10

AI Summary

This work addresses the limitations of traditional greedy algorithms, which yield suboptimal regression trees, and dynamic programming approaches, which incur prohibitive computational costs. To overcome these challenges, the authors propose an efficient near-optimal algorithm that integrates lookahead search with rank-one Cholesky updates of the Gram matrix, augmented by dynamic programming acceleration and sparse modeling techniques. This novel combination significantly enhances the training efficiency and scalability of piecewise linear regression trees. The method achieves substantial computational speedups over existing approaches while preserving model sparsity and delivering prediction accuracy close to the optimal solution.

Abstract

Regression trees are among the most interpretable yet expressive model classes in machine learning. Historically, greedy induction has been the dominant approach for constructing well-performing regression trees. While optimal methods based on dynamic programming and branch-and-bound exist, they are computationally prohibitive for general linear regression trees, despite often achieving substantially better performance than greedy approaches. Recent work has shown that specialized lookahead strategies can dramatically improve runtime while maintaining near-optimal performance, primarily in classification settings. In this work, we develop a novel algorithm for near-optimal, sparse, piecewise linear regression trees that combines a lookahead-style search strategy with efficient rank-one Cholesky updates of the Gram matrix. We demonstrate, both theoretically and empirically, that our method achieves a favorable trade-off between computational efficiency, predictive accuracy, and sparsity, and scales significantly better than the current state of the art.

Problem

• regression trees
• piecewise linear
• computational efficiency
• sparsity
• interpretability

Innovation

• Cholesky update
• lookahead search
• piecewise linear regression trees
• near-optimal tree induction
• Gram matrix