IMPROVE: Impact of Mobile Phones on Remote Online Virtual Education

2024-12-13

arXiv.org

AI Summary

This study systematically investigates the impact of smartphone usage on learners’ cognitive and behavioral responses during online remote education. To address critical gaps—namely, the scarcity of ecologically valid data and insufficient multimodal annotation—we introduce IMPROVE, the first open-source multimodal dataset (N=120 participants across three interaction-intensity conditions), capturing synchronized 16-channel data including EEG, eye-tracking, RGB/IR video, IMU, and head-pose signals. We further propose a semi-automated EEG–behavioral signal fusion method for high-fidelity event re-annotation. For the first time, we achieve precise temporal alignment between smartphone usage events and multidimensional cognitive metrics—including academic performance, subjective self-reports, and physiological responses—under both controlled and naturalistic settings. Analyses reveal significant smartphone-induced modulations in α/θ EEG power spectra and gaze distribution patterns. The IMPROVE dataset and annotation framework provide a rigorous empirical foundation for designing educational technology interventions and advancing cognitive load modeling.

Technology Category

• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Security and Privacy: Data transparency and provenance

Abstract

This work presents the IMPROVE dataset, designed to evaluate the effects of mobile phone usage on learners during online education. The dataset not only assesses academic performance and subjective learner feedback but also captures biometric, behavioral, and physiological signals, providing a comprehensive analysis of the impact of mobile phone use on learning. Multimodal data were collected from 120 learners in three groups with different phone interaction levels. A setup involving 16 sensors was implemented to collect data that have proven to be effective indicators for understanding learner behavior and cognition, including electroencephalography waves, videos, eye tracker, etc. The dataset includes metadata from the processed videos like face bounding boxes, facial landmarks, and Euler angles for head pose estimation. In addition, learner performance data and self-reported forms are included. Phone usage events were labeled, covering both supervisor-triggered and uncontrolled events. A semi-manual re-labeling system, using head pose and eye tracker data, is proposed to improve labeling accuracy. Technical validation confirmed signal quality, with statistical analyses revealing biometric changes during phone use.

Problem

• Evaluates mobile phone impact on online education learners
• Analyzes behavioral and biometric changes during phone usage
• Provides multimodal dataset for nomophobia and academic performance research

Innovation

• Multimodal sensor setup monitors learner activity
• Semi-supervised labeling for phone usage events
• Public dataset with synchronized multi-platform recordings