ForestFormer3D: A Unified Framework for End-to-End Segmentation of Forest LiDAR 3D Point Clouds

2025-06-20

AI Summary

To address the challenges of structural complexity and poor generalization in joint individual tree and semantic segmentation of natural forest LiDAR point clouds, this paper proposes the first unified Transformer-based framework. Methodologically, we introduce three novel components: (1) ISA-guided query point selection, (2) score-driven block merging inference, and (3) one-to-many association training. These are integrated with geometry-aware query initialization, dynamic block aggregation, contrastive point-to-instance matching, and multi-scale feature interaction. Our method achieves state-of-the-art performance on the FOR-instanceV2 benchmark for individual tree segmentation. Cross-domain evaluation on Wytham Woods and LAUTx demonstrates strong generalization across diverse forest types and LiDAR sensor modalities. The framework significantly improves both end-to-end joint segmentation accuracy and robustness, setting a new standard for unified forest point cloud parsing.

Technology Category

• Computer Vision: Segmentation
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

The segmentation of forest LiDAR 3D point clouds, including both individual tree and semantic segmentation, is fundamental for advancing forest management and ecological research. However, current approaches often struggle with the complexity and variability of natural forest environments. We present ForestFormer3D, a new unified and end-to-end framework designed for precise individual tree and semantic segmentation. ForestFormer3D incorporates ISA-guided query point selection, a score-based block merging strategy during inference, and a one-to-many association mechanism for effective training. By combining these new components, our model achieves state-of-the-art performance for individual tree segmentation on the newly introduced FOR-instanceV2 dataset, which spans diverse forest types and regions. Additionally, ForestFormer3D generalizes well to unseen test sets (Wytham woods and LAUTx), showcasing its robustness across different forest conditions and sensor modalities. The FOR-instanceV2 dataset and the ForestFormer3D code will be released soon.

Problem

• Segmentation of complex forest LiDAR 3D point clouds
• Improving accuracy in individual tree and semantic segmentation
• Handling variability in natural forest environments

Innovation

• ISA-guided query point selection
• Score-based block merging strategy
• One-to-many association training mechanism