Zero-Shot Tree Detection and Segmentation from Aerial Forest Imagery

2025-06-03

AI Summary

To address the heavy reliance on annotated data and poor scalability in tree detection and segmentation from large-scale aerial forest imagery, this paper proposes the first zero-shot single-tree detection and segmentation framework tailored for forestry remote sensing. Methodologically, we introduce Segment Anything Model 2 (SAM2) to forestry for the first time, establishing a “detection output → spatial prompts → zero-shot segmentation” transfer paradigm: existing tree detection models’ outputs serve as spatial prompts for SAM2, eliminating the need for any training or fine-tuning. Experiments across multiple aerial image datasets demonstrate that our approach achieves near-supervised segmentation accuracy under zero-shot conditions (mAP@0.5 = 0.72), substantially reducing annotation dependency. The implementation is open-sourced and designed for plug-and-play ecological monitoring applications.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Large-scale delineation of individual trees from remote sensing imagery is crucial to the advancement of ecological research, particularly as climate change and other environmental factors rapidly transform forest landscapes across the world. Current RGB tree segmentation methods rely on training specialized machine learning models with labeled tree datasets. While these learning-based approaches can outperform manual data collection when accurate, the existing models still depend on training data that's hard to scale. In this paper, we investigate the efficacy of using a state-of-the-art image segmentation model, Segment Anything Model 2 (SAM2), in a zero-shot manner for individual tree detection and segmentation. We evaluate a pretrained SAM2 model on two tasks in this domain: (1) zero-shot segmentation and (2) zero-shot transfer by using predictions from an existing tree detection model as prompts. Our results suggest that SAM2 not only has impressive generalization capabilities, but also can form a natural synergy with specialized methods trained on in-domain labeled data. We find that applying large pretrained models to problems in remote sensing is a promising avenue for future progress. We make our code available at: https://github.com/open-forest-observatory/tree-detection-framework.

Problem

• Detect and segment individual trees from aerial imagery without training data
• Evaluate zero-shot segmentation using SAM2 for forest monitoring
• Combine pretrained models with specialized methods for improved accuracy

Innovation

• Zero-shot segmentation with SAM2 model
• Transfer learning using existing tree predictions
• Pretrained models for remote sensing tasks