M2LADS Demo: A System for Generating Multimodal Learning Analytics Dashboards

2025-02-21

AI Summary

This study addresses key challenges in learning analytics—namely, the difficulty of integrating multimodal physiological and behavioral data (e.g., EEG, heart rate, eye-tracking, video, and task logs), low temporal synchronization accuracy, and inefficient re-annotation. To this end, we propose the first full-modality temporal alignment and activity-driven visualization framework specifically designed for learning analytics. Our method introduces a millisecond-precision multi-source time-synchronization algorithm, a biometric signal preprocessing pipeline, and a streaming visualization engine, implemented as a React-based web system. The framework enables automatic alignment and interpretable re-annotation of five modalities. It delivers an interactive dashboard covering 12+ analytical dimensions—including attention, cognitive load, and visual focus—significantly improving data cleaning and annotation efficiency. The system provides data scientists with a unified, traceable, and interpretable panoramic view of learning processes.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We present a demonstration of a web-based system called M2LADS ("System for Generating Multimodal Learning Analytics Dashboards"), designed to integrate, synchronize, visualize, and analyze multimodal data recorded during computer-based learning sessions with biosensors. This system presents a range of biometric and behavioral data on web-based dashboards, providing detailed insights into various physiological and activity-based metrics. The multimodal data visualized include electroencephalogram (EEG) data for assessing attention and brain activity, heart rate metrics, eye-tracking data to measure visual attention, webcam video recordings, and activity logs of the monitored tasks. M2LADS aims to assist data scientists in two key ways: (1) by providing a comprehensive view of participants' experiences, displaying all data categorized by the activities in which participants are engaged, and (2) by synchronizing all biosignals and videos, facilitating easier data relabeling if any activity information contains errors.

Problem

• Integrate multimodal learning data
• Visualize biometric and behavioral metrics
• Synchronize biosignals for error correction

Innovation

• Web-based multimodal data integration
• Synchronized biosignal and video analysis
• Comprehensive physiological and behavioral visualization