Multi-Modal Landslide Detection from Sentinel-1 SAR and Sentinel-2 Optical Imagery Using Multi-Encoder Vision Transformers and Ensemble Learning

2026-04-07

AI Summary

This study addresses the challenge of efficiently and accurately detecting landslides when pre-disaster optical imagery is unavailable by proposing a non-classical change detection paradigm that eliminates the need for pre-event optical data. The method integrates Sentinel-1 SAR and Sentinel-2 optical imagery, employing a multi-encoder Vision Transformer to process multimodal inputs separately, followed by an ensemble learning framework that combines deep neural networks with gradient boosting models (LightGBM/XGBoost). This architecture supports flexible deployment with either single-modality or joint inputs. Experimental results demonstrate that the proposed approach achieves an F1 score of 0.919 in landslide detection, striking an effective balance between precision and recall. It significantly outperforms existing methods, confirming its effectiveness, robustness, and scalability.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Landslides represent a major geohazard with severe impacts on human life, infrastructure, and ecosystems, underscoring the need for accurate and timely detection approaches to support disaster risk reduction. This study proposes a modular, multi-model framework that fuses Sentinel-2 optical imagery with Sentinel-1 Synthetic Aperture Radar (SAR) data, for robust landslide detection. The methodology leverages multi-encoder vision transformers, where each data modality is processed through separate lightweight pretrained encoders, achieving strong performance in landslide detection. In addition, the integration of multiple models, particularly the combination of neural networks and gradient boosting models (LightGBM and XGBoost), demonstrates the power of ensemble learning to further enhance accuracy and robustness. Derived spectral indices, such as NDVI, are integrated alongside original bands to enhance sensitivity to vegetation and surface changes. The proposed methodology achieves a state-of-the-art F1 score of 0.919 on landslide detection, addressing a patch-based classification task rather than pixel-level segmentation and operating without pre-event Sentinel-2 data, highlighting its effectiveness in a non-classical change detection setting. It also demonstrated top performance in a machine learning competition, achieving a strong balance between precision and recall and highlighting the advantages of explicitly leveraging the complementary strengths of optical and radar data. The conducted experiments and research also emphasize scalability and operational applicability, enabling flexible configurations with optical-only, SAR-only, or combined inputs, and offering a transferable framework for broader natural hazard monitoring and environmental change applications. Full training and inference code can be found in https://github.com/IoannisNasios/sentinel-landslide-cls.

Problem

• landslide detection
• multi-modal remote sensing
• Sentinel-1 SAR
• Sentinel-2 optical imagery
• disaster risk reduction

Innovation

• multi-encoder Vision Transformers
• ensemble learning
• multi-modal fusion
• landslide detection
• Sentinel-1/2