A Silent Speech Decoding System from EEG and EMG with Heterogenous Electrode Configurations

2025-06-16

AI Summary

This study addresses the challenge of silent speech decoding for aphasic patients, where poor model generalizability arises from inconsistent EEG/EMG signal acquisition across heterogeneous electrode configurations. We propose the first multi-task neural architecture supporting variable electrode layouts, integrating heterogeneous EEG/EMG signal alignment, cross-subject transfer learning, and multi-task optimization to achieve robust, subject- and language-agnostic decoding. Evaluated on healthy subjects and aphasic patients, our method achieves word-level classification accuracies of 95.3% and 54.5%, respectively—substantially surpassing subject-specific baselines (70.1% and 13.2%). It also improves cross-lingual calibration efficiency and stability. By enabling high generalizability with minimal invasiveness, this work establishes a novel clinical paradigm for silent speech interfaces tailored to neurologically impaired populations.

Technology Category

• Machine Learning: Multimodal Learning
• Natural Language Processing: Speech
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Silent speech decoding, which performs unvocalized human speech recognition from electroencephalography/electromyography (EEG/EMG), increases accessibility for speech-impaired humans. However, data collection is difficult and performed using varying experimental setups, making it nontrivial to collect a large, homogeneous dataset. In this study we introduce neural networks that can handle EEG/EMG with heterogeneous electrode placements and show strong performance in silent speech decoding via multi-task training on large-scale EEG/EMG datasets. We achieve improved word classification accuracy in both healthy participants (95.3%), and a speech-impaired patient (54.5%), substantially outperforming models trained on single-subject data (70.1% and 13.2%). Moreover, our models also show gains in cross-language calibration performance. This increase in accuracy suggests the feasibility of developing practical silent speech decoding systems, particularly for speech-impaired patients.

Problem

• Decoding silent speech from EEG/EMG with varied electrode setups
• Handling heterogeneous datasets for improved speech recognition
• Enhancing accuracy for speech-impaired patients via multi-task learning

Innovation

• Neural networks handle heterogeneous EEG/EMG electrode placements
• Multi-task training on large-scale EEG/EMG datasets
• Improved cross-language calibration performance