🤖 AI Summary
To address the limited generalization capability of k-nearest neighbors (k-NN) ensemble methods, this paper proposes an adaptive k-NN classifier based on discriminative subspace projection, embedded within the Bootstrap aggregating (bagging) framework. For each base classifier, trained on a bootstrap sample, we jointly learn an optimal discriminative projection direction and adaptively select the neighborhood size k—thereby simultaneously enhancing discriminability and ensemble diversity. Extensive experiments across multiple benchmark datasets demonstrate that the proposed ensemble significantly outperforms random forests and state-of-the-art k-NN ensembles. An open-source R package ensures reproducibility and practical applicability. The key contributions are: (1) the first k-NN ensemble framework integrating discriminative subspace learning with adaptive k-selection; and (2) a systematic improvement in the generalization performance of k-NN within bagging, achieved by explicitly increasing both the effectiveness and dissimilarity of base classifiers.
📝 Abstract
In this paper we introduce a simple and intuitive adaptive k nearest neighbours classifier, and explore its utility within the context of bootstrap aggregating ("bagging"). The approach is based on finding discriminant subspaces which are computationally efficient to compute, and are motivated by enhancing the discrimination of classes through nearest neighbour classifiers. This adaptiveness promotes diversity of the individual classifiers fit across different bootstrap samples, and so further leverages the variance reducing effect of bagging. Extensive experimental results are presented documenting the strong performance of the proposed approach in comparison with Random Forest classifiers, as well as other nearest neighbours based ensembles from the literature, plus other relevant benchmarks. Code to implement the proposed approach is available in the form of an R package from https://github.com/DavidHofmeyr/BOPNN.