Technology Category
Intelligent Robots: Multimodal Perception & Sensor FusionComputer Vision: Multi-modal VisionMachine Learning: Multimodal Learning
Sleep Stage Classification using Multimodal Embedding Fusion from EOG and PSM
🤖 AI Summary
Conventional polysomnography (PSG) for home sleep monitoring relies heavily on electroencephalography (EEG), entailing complex, intrusive hardware and limiting practical deployment.
Method: This paper proposes a non-invasive, five-stage sleep staging framework integrating electrooculography (EOG) and pressure-sensitive mat (PSM) signals. It introduces ImageBind—a multimodal pretrained model—for cross-modal EOG-PSM fusion, jointly modeling dual-channel EOG time series and PSM-derived pressure spatial-temporal representations, enabling zero- or few-shot fine-tuning.
Contribution/Results: Evaluated on 85 full-night clinical recordings, the method significantly outperforms DeepSleepNet, ViViT, and MBT. After fine-tuning, accuracy approaches EEG-based baselines; without fine-tuning, it demonstrates strong generalization across subjects. The approach establishes a novel paradigm for low-burden, highly adaptable home sleep assessment—eliminating EEG dependency while preserving staging fidelity.
Technology Category
Application Category
📝 Abstract
Accurate sleep stage classification is essential for diagnosing sleep disorders, particularly in aging populations. While traditional polysomnography (PSG) relies on electroencephalography (EEG) as the gold standard, its complexity and need for specialized equipment make home-based sleep monitoring challenging. To address this limitation, we investigate the use of electrooculography (EOG) and pressure-sensitive mats (PSM) as less obtrusive alternatives for five-stage sleep-wake classification. This study introduces a novel approach that leverages ImageBind, a multimodal embedding deep learning model, to integrate PSM data with dual-channel EOG signals for sleep stage classification. Our method is the first reported approach that fuses PSM and EOG data for sleep stage classification with ImageBind. Our results demonstrate that fine-tuning ImageBind significantly improves classification accuracy, outperforming existing models based on single-channel EOG (DeepSleepNet), exclusively PSM data (ViViT), and other multimodal deep learning approaches (MBT). Notably, the model also achieved strong performance without fine-tuning, highlighting its adaptability to specific tasks with limited labeled data, making it particularly advantageous for medical applications. We evaluated our method using 85 nights of patient recordings from a sleep clinic. Our findings suggest that pre-trained multimodal embedding models, even those originally developed for non-medical domains, can be effectively adapted for sleep staging, with accuracies approaching systems that require complex EEG data.
Problem
Research questions and friction points this paper is trying to address.
Classify sleep stages using EOG and PSM data
Improve home-based sleep monitoring accuracy
Adapt multimodal deep learning for medical use
Innovation
Methods, ideas, or system contributions that make the work stand out.
Fuses EOG and PSM data with ImageBind
Improves accuracy via multimodal embedding
Adapts pre-trained model for sleep staging
🔎 Similar Papers
No similar papers found.
