On What Depends the Robustness of Multi-source Models to Missing Data in Earth Observation?

2025-03-25

AI Summary

This study investigates the robustness mechanisms of multi-source remote sensing models under scenarios where one source is missing or only a single source is available. To address performance degradation caused by data incompleteness in Earth observation (EO), we systematically evaluate six state-of-the-art multi-source models through controlled ablation experiments, quantitative analysis, and attribution methods. Our analysis reveals that robustness emerges from the interplay among task characteristics, cross-sensor complementarity, and model architecture. Notably, we identify for the first time that strategically removing redundant modalities can improve prediction accuracy—challenging the conventional “more data is always better” assumption. These findings provide empirical foundations for optimizing EO data acquisition strategies and designing lightweight, reliable models. The work advances the development of efficient and robust intelligent remote sensing systems.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Data transparency and provenance

Abstract

In recent years, the development of robust multi-source models has emerged in the Earth Observation (EO) field. These are models that leverage data from diverse sources to improve predictive accuracy when there is missing data. Despite these advancements, the factors influencing the varying effectiveness of such models remain poorly understood. In this study, we evaluate the predictive performance of six state-of-the-art multi-source models in predicting scenarios where either a single data source is missing or only a single source is available. Our analysis reveals that the efficacy of these models is intricately tied to the nature of the task, the complementarity among data sources, and the model design. Surprisingly, we observe instances where the removal of certain data sources leads to improved predictive performance, challenging the assumption that incorporating all available data is always beneficial. These findings prompt critical reflections on model complexity and the necessity of all collected data sources, potentially shaping the way for more streamlined approaches in EO applications.

Problem

• Factors affecting multi-source model robustness to missing Earth Observation data
• Evaluating model performance with single missing or available data sources
• Impact of task nature, data complementarity, and model design on efficacy

Innovation

• Evaluates six multi-source models robustness
• Links efficacy to task nature and data complementarity
• Challenges necessity of all data sources