EEG-to-Text Translation: A Model for Deciphering Human Brain Activity

2025-05-20

AI Summary

This study addresses the low accuracy and poor robustness of semantic EEG-to-text decoding in non-invasive brain–computer interfaces. We propose the first end-to-end architecture integrating a bidirectional LSTM encoder with a pretrained Transformer decoder. Our approach innovatively incorporates time-frequency EEG feature embeddings to enhance long-range temporal modeling and ensure consistency between neural dynamics and semantic generation. Evaluated on a standard EEG-to-text benchmark, our method achieves ROUGE-1 of 38.00% (a 9-percentage-point improvement over T5), ROUGE-L F1 of 32.51%, character error rate (CER) of 0.5795, and word error rate (WER) of 0.7280—outperforming all existing approaches. These results mark a significant advance in interpretable, natural-language decoding from neural activity.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

With the rapid advancement of large language models like Gemini, GPT, and others, bridging the gap between the human brain and language processing has become an important area of focus. To address this challenge, researchers have developed various models to decode EEG signals into text. However, these models still face significant performance limitations. To overcome these shortcomings, we propose a new model, R1 Translator, which aims to improve the performance of EEG-to-text decoding. The R1 Translator model combines a bidirectional LSTM encoder with a pretrained transformer-based decoder, utilizing EEG features to produce high-quality text outputs. The model processes EEG embeddings through the LSTM to capture sequential dependencies, which are then fed into the transformer decoder for effective text generation. The R1 Translator excels in ROUGE metrics, outperforming both T5 (previous research) and Brain Translator. Specifically, R1 achieves a ROUGE-1 score of 38.00% (P), which is up to 9% higher than T5 (34.89%) and 3% better than Brain (35.69%). It also leads in ROUGE-L, with a F1 score of 32.51%, outperforming T5 by 3% (29.67%) and Brain by 2% (30.38%). In terms of CER, R1 achieves a CER of 0.5795, which is 2% lower than T5 (0.5917) and 4% lower than Brain (0.6001). Additionally, R1 performs better in WER with a score of 0.7280, outperforming T5 by 4.3% (0.7610) and Brain by 3.6% (0.7553). Code is available at https://github.com/Mmurrad/EEG-To-text.

Problem

• Decoding EEG signals into accurate text outputs
• Improving EEG-to-text translation model performance
• Overcoming limitations in existing EEG-to-text models

Innovation

• Combines bidirectional LSTM with transformer decoder
• Utilizes EEG embeddings for text generation
• Achieves higher ROUGE scores than previous models