A review on data fusion in multimodal learning analytics and educational data mining

2025-11-25

AI Summary

This study addresses the challenges of fusing heterogeneous educational data—such as audio, video, eye-tracking, physiological signals, and behavioral logs—in multimodal learning analytics (MLA), and the consequent lack of robust intervention support. We systematically review and establish a taxonomy and technical pathway for data fusion in educational contexts. Methodologically, we propose a novel three-tier fusion framework spanning feature-level, decision-level, and model-level integration, synergizing machine learning and educational data mining techniques to enhance cross-modal collaborative modeling. Our analysis identifies critical bottlenecks in temporal alignment, interpretability, and real-time intervention capability, and clarifies a theoretical paradigm and developmental roadmap for data fusion in intelligent learning environments. Results demonstrate that principled multimodal fusion significantly improves learning state recognition accuracy and the efficacy of pedagogical interventions, thereby providing a methodological foundation for next-generation adaptive learning systems.

Technology Category

• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

The new educational models such as smart learning environments use of digital and context-aware devices to facilitate the learning process. In this new educational scenario, a huge quantity of multimodal students' data from a variety of different sources can be captured, fused, and analyze. It offers to researchers and educators a unique opportunity of being able to discover new knowledge to better understand the learning process and to intervene if necessary. However, it is necessary to apply correctly data fusion approaches and techniques in order to combine various sources of multimodal learning analytics (MLA). These sources or modalities in MLA include audio, video, electrodermal activity data, eye-tracking, user logs, and click-stream data, but also learning artifacts and more natural human signals such as gestures, gaze, speech, or writing. This survey introduces data fusion in learning analytics (LA) and educational data mining (EDM) and how these data fusion techniques have been applied in smart learning. It shows the current state of the art by reviewing the main publications, the main type of fused educational data, and the data fusion approaches and techniques used in EDM/LA, as well as the main open problems, trends, and challenges in this specific research area.

Problem

• Reviewing data fusion approaches for multimodal learning analytics and educational data mining
• Analyzing how to combine diverse educational data sources like audio, video, and eye-tracking
• Identifying challenges in fusing multimodal student data from smart learning environments

Innovation

• Data fusion for multimodal learning analytics
• Combining diverse educational data sources
• Survey of fusion techniques in smart learning