Multimodal Object Detection Under Sparse Forest-Canopy Occlusion

2026-05-14

AI Summary

This study addresses the challenge of detecting human targets in forested environments, where sparse canopy cover, complex structural clutter, and viewpoint occlusions severely degrade detection performance. To overcome these limitations, the authors propose a multimodal fusion approach that integrates LiDAR-based vegetation penetration analysis, multiscale sparse representation fusion of visible and thermal infrared imagery, and airborne optical sectioning (AOS). Notably, this work presents the first synergistic application of visible–thermal fusion with AOS for understory target detection. Building upon YOLOv5, the method is fine-tuned on thermal and fused imagery, achieving an mAP of approximately 0.83 on the top three classes of the FLIR dataset. The proposed framework establishes an effective multimodal perception baseline for UAV-based search-and-rescue and surveillance operations in dense forest settings.

Abstract

Reliable detection of humans beneath forest canopy remains a difficult remote-sensing challenge due to sparse, structured, and viewpoint-dependent occlusion. This paper presents a multimodal proof-of-concept pipeline that integrates three complementary approaches: (i) experimental evaluation of LiDAR returns through vegetation to assess the feasibility of active sensing, (ii) visible--thermal image fusion using a multi-scale transform and sparse-representation framework to enhance human saliency, and (iii) synthetic-aperture image formation via Airborne Optical Sectioning (AOS) to suppress canopy clutter. A YOLOv5 detector is fine-tuned on the Teledyne FLIR thermal dataset and evaluated on thermal and fused imagery. Results show that the tested terrestrial LiDAR configuration provides limited penetration for object-level detection, while visible--thermal fusion improves target visibility in low-contrast scenes and AOS enhances ground-plane detection in synthetic forest imagery. The fine-tuned YOLOv5 achieves a mean average precision of $\sim$0.83 on the top three FLIR classes. These findings establish an initial baseline for UAV-deployable search-and-rescue and surveillance systems operating in forested environments, and motivate future work on dedicated forest datasets and real-time multimodal integration.

Problem

• Multimodal Object Detection
• Forest-Canopy Occlusion
• Remote Sensing
• Human Detection
• Sparse Occlusion

Innovation

• multimodal fusion
• forest-canopy occlusion
• Airborne Optical Sectioning
• LiDAR penetration
• visible-thermal imaging