Technology Category
Intelligent Robots: Multimodal Perception & Sensor FusionMachine Learning: Multimodal LearningHumans and AI: Brain-Sensing and Analysis
Beyond Hearing: Learning Task-agnostic ExG Representations from Earphones via Physiology-informed Tokenization
🤖 AI Summary
To address two key bottlenecks in ear-worn electrophysiological (ExG) signal modeling—data scarcity (largely due to controlled lab settings) and poor model generalizability (stemming from task-specific architectures)—this paper introduces the first foundational model for ear-worn ExG signals tailored to free-living scenarios. We develop a custom ear-worn hardware platform to collect the 50-hour multimodal DailySense dataset under naturalistic conditions. We propose Physiological-information-guided Multi-band Tokenization (PiMT), a novel tokenization method that enables cross-task, adaptive full-spectrum representation learning by leveraging physiological priors. PiMT is integrated into an autoencoder-based self-supervised pretraining framework. Extensive evaluation on DailySense and four public benchmarks demonstrates its effectiveness. Results show PiMT consistently outperforms state-of-the-art methods across five sensory tasks, significantly enhancing representation robustness and multi-task generalization capability.
Technology Category
Application Category
📝 Abstract
Electrophysiological (ExG) signals offer valuable insights into human physiology, yet building foundation models that generalize across everyday tasks remains challenging due to two key limitations: (i) insufficient data diversity, as most ExG recordings are collected in controlled labs with bulky, expensive devices; and (ii) task-specific model designs that require tailored processing (i.e., targeted frequency filters) and architectures, which limit generalization across tasks. To address these challenges, we introduce an approach for scalable, task-agnostic ExG monitoring in the wild. We collected 50 hours of unobtrusive free-living ExG data with an earphone-based hardware prototype to narrow the data diversity gap. At the core of our approach is Physiology-informed Multi-band Tokenization (PiMT), which decomposes ExG signals into 12 physiology-informed tokens, followed by a reconstruction task to learn robust representations. This enables adaptive feature recognition across the full frequency spectrum while capturing task-relevant information. Experiments on our new DailySense dataset-the first to enable ExG-based analysis across five human senses-together with four public ExG benchmarks, demonstrate that PiMT consistently outperforms state-of-the-art methods across diverse tasks.
Problem
Research questions and friction points this paper is trying to address.
Developing task-agnostic ExG monitoring for real-world applications
Addressing limited data diversity in electrophysiological signal collection
Overcoming task-specific model limitations for cross-task generalization
Innovation
Methods, ideas, or system contributions that make the work stand out.
Earphone-based hardware collects free-living ExG data
Physiology-informed tokenization creates 12 signal tokens
Reconstruction task learns robust task-agnostic representations
🔎 Similar Papers
No similar papers found.
