Pitfalls of Unlabeled Disagreement-Based Drift Detection in Streaming Tree Ensembles

2026-05-12

AI Summary

This study addresses the limitations of existing disagreement-based concept drift detectors in unlabeled data streams, which are prone to interference from benign distributional shifts and exhibit limited efficacy within incremental decision tree ensembles. The authors propose a novel approach that introduces label flipping among ensemble members to construct batch-specific disagreement measures for drift detection. Their analysis reveals a fundamental limitation of incremental decision trees: structural rigidity impedes their ability to adequately reflect learning potential. The work further suggests that rule decomposition–based reconstruction offers a promising direction for enhancing model adaptability. Empirical results demonstrate that the proposed method performs well in multilayer perceptron ensembles but is significantly outperformed by loss-based detectors when applied to incremental decision tree ensembles.

Abstract

Detecting concept drift in high-speed data streams remains challenging, particularly when models must operate on unlabeled data and avoid false alarms caused by benign shifts. While disagreement-based uncertainty has shown promise in neural networks, its adaptation to ensembles of incremental decision trees (IDTs) remains largely unexplored. We investigate this approach by constructing batch-specific disagreement measures via label flipping in ensemble members and evaluating their effectiveness for drift detection in tabular data streams. Our experiments show that, although this method performs well in ensembles of multi-layer perceptrons (MLPs), it consistently underperforms loss-based detectors when applied to IDTs. We attribute this behavior to the intrinsic rigidity of IDTs: learning primarily through structural expansion, with limited parameter adaptation, restricts model plasticity and prevents disagreement from reliably reflecting learning potential. Recent work on restructuring IDTs using their intrinsic decomposition into non-overlapping rules offers a promising direction for improving adaptability.

Problem

• concept drift
• unlabeled data
• disagreement-based detection
• incremental decision trees
• streaming data

Innovation

• disagreement-based drift detection
• incremental decision trees
• concept drift
• model plasticity
• unlabeled data streams