How Well Can We Decode Vowels from Auditory EEG - A Rigorous Cross-Subject Benchmark with Honest Assessment

2026-04-22

arXiv.org

AI Summary

This study systematically evaluates whether auditory-evoked EEG signals can support cross-subject decoding of five vowel percepts. Leveraging the OpenNeuro dataset ds006104, we establish the first end-to-end reproducible benchmark framework that integrates trial-level source localization, subject-level inference, and multiple comparison correction, assessing the decoding performance of 13 models—including random forests, deep neural networks, and Riemannian minimum distance to mean (MDM)—via leave-one-subject-out cross-validation. Results reveal that the best-performing model achieves only 21.47% balanced accuracy, barely above the 20% chance level, with no method surviving multiple comparison correction. Inter-subject variability accounts for 72.24% of the variance—substantially exceeding the effect of vowel category (2.04%)—and increasing the number of training subjects does not yield monotonic performance gains, collectively indicating limited evidence for reliable cross-subject vowel decoding from current EEG-based approaches.

Abstract

EEG based phoneme decoding is promising for brain computer interfaces, but many prior studies rely on within subject evaluation, small cohorts, or weak leakage control. We present a reproducible cross subject benchmark for five class vowel decoding (a, e, i, o, u) from auditory EEG using OpenNeuro ds006104 (16 subjects, 61 channels, 256 Hz). Under strict leave one subject out evaluation with training only normalization and explicit anti leakage checks, we compare 14 pipelines from classical machine learning, deep learning, and Riemannian methods. The best full feature model (XGBoost) reaches 24.5 percent accuracy (chance 20 percent), while differential entropy features with LightGBM reach 25.5 percent in feature specific analysis. After multiple comparison correction, strong pairwise model advantages are limited. Classical methods are competitive with deep models in this low signal regime. Additional analyses (ablation, pairwise vowels, within subject CV, ERP, temporal generalization, and electrode importance) indicate that vowel information is real but weak and mainly carried by early transient auditory responses. We release code and evaluation scripts for full reproducibility.

Problem

• cross-subject decoding
• auditory-evoked EEG
• vowel perception
• benchmarking
• leakage control

Innovation

• leakage-audited benchmarking
• cross-subject EEG decoding
• auditory-evoked potentials
• vowel perception
• reproducible neuroscience