Mind the Missing Split: Resolving Feature Heterogeneity in Swarm Learning with Random Forests

2026-07-28

AI Summary

This work addresses the challenge in Swarm Learning where partially overlapping feature spaces among participants lead to missing splitting features during random forest inference. To overcome this limitation, the study proposes a novel inference approach that integrates deterministic and probabilistic strategies within the Swarm Learning framework, effectively leveraging participant-specific local features rather than relying solely on shared features. The method enables distributed training and inference with random forests and demonstrates significant performance gains over baselines that use only common features as well as over locally trained standalone models across nine benchmark datasets. By harnessing heterogeneous feature information, the proposed approach enhances model generalization and offers a robust solution to a key obstacle in feature-heterogeneous collaborative learning settings.

Abstract

Swarm Learning is a decentralized collaborative learning mechanism that allows multiple organizations to train a shared model without central coordination or direct data sharing. In typical horizontal Swarm Learning, datasets across sites are usually assumed to share the same feature set. However, in real-world applications, sites often have partially overlapping features because measurements, protocols, and available covariates differ across sites. This feature heterogeneity creates a practical issue for machine learning algorithms such as Random Forests. Specifically, when decision trees are pooled into a global Random Forest, inference at a given site can become ill-defined if a traversal encounters a split on a feature that is not available locally, often forcing organizations to discard site-specific variables upfront. In this paper, we address feature heterogeneity in Swarm Learning with Random Forests under partially overlapping feature spaces. We propose several deterministic and probabilistic inference-time strategies that resolve such missing splits without restricting training to the intersection of features. We evaluate the methods on nine datasets and demonstrate that they outperform both the intersection baseline and locally trained models across a broad range of scenarios.

Problem

• Swarm Learning
• feature heterogeneity
• Random Forests
• missing features
• decentralized learning

Innovation

• Swarm Learning
• Feature Heterogeneity
• Random Forests
• Missing Split
• Decentralized Learning