Multi-modal Data Fusion and Deep Ensemble Learning for Accurate Crop Yield Prediction

2025-02-09

AI Summary

To address insufficient accuracy in crop yield prediction, this paper proposes RicEns-Net, a multimodal deep ensemble model. It is the first to jointly integrate Sentinel-1/2/3 SAR and optical remote sensing data with meteorological observations—including land surface temperature and precipitation—constructing a compact 15-dimensional feature set from over 100 raw features across five modalities. A novel multimodal feature selection mechanism and a dedicated deep ensemble architecture are designed to mitigate the curse of dimensionality and enhance generalization. Evaluated on the EY Open Science Challenge 2023 benchmark dataset, RicEns-Net achieves a mean absolute error (MAE) of 341 kg/ha—approximately 5–6% of the regional minimum average yield—significantly outperforming existing state-of-the-art methods. This demonstrates the efficacy of synergistic modeling with heterogeneous multi-source data and lightweight feature engineering in agricultural remote sensing yield forecasting.

Technology Category

Machine Learning: Ensemble MethodsIntelligent Robots: Multimodal Perception & Sensor FusionComputer Vision: Multi-modal Vision

Application Category

Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAGSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsWeb Mining and Content Analysis: Large pretrained models with web data

Abstract

This study introduces RicEns-Net, a novel Deep Ensemble model designed to predict crop yields by integrating diverse data sources through multimodal data fusion techniques. The research focuses specifically on the use of synthetic aperture radar (SAR), optical remote sensing data from Sentinel 1, 2, and 3 satellites, and meteorological measurements such as surface temperature and rainfall. The initial field data for the study were acquired through Ernst&Young's (EY) Open Science Challenge 2023. The primary objective is to enhance the precision of crop yield prediction by developing a machine-learning framework capable of handling complex environmental data. A comprehensive data engineering process was employed to select the most informative features from over 100 potential predictors, reducing the set to 15 features from 5 distinct modalities. This step mitigates the ``curse of dimensionality"and enhances model performance. The RicEns-Net architecture combines multiple machine learning algorithms in a deep ensemble framework, integrating the strengths of each technique to improve predictive accuracy. Experimental results demonstrate that RicEns-Net achieves a mean absolute error (MAE) of 341 kg/Ha (roughly corresponds to 5-6% of the lowest average yield in the region), significantly exceeding the performance of previous state-of-the-art models, including those developed during the EY challenge.

Problem

• Multi-modal data fusion for crop yield prediction
• Deep ensemble learning with diverse data sources
• Enhancing precision in agricultural yield forecasting

Innovation

• Multi-modal data fusion
• Deep Ensemble Learning
• Feature selection optimization