Mentality: A Mamba-based Approach towards Foundation Models for EEG

2025-09-02

AI Summary

EEG signals exhibit strong noise, high dimensionality, nonlinearity, and complex spatiotemporal dynamics, limiting the modeling capacity of conventional methods. To address this, this work introduces the Mamba architecture—the first application of a state-space model (SSM) to EEG foundation modeling—leveraging its selective state-space mechanism to efficiently capture long-range spatiotemporal dependencies. We propose a two-stage paradigm: self-supervised reconstruction pretraining followed by supervised fine-tuning, enabling scalable learning on large-scale EEG data and enhancing generalization. Experiments demonstrate that the resulting model achieves 0.72 AUROC on out-of-distribution seizure detection, significantly outperforming mainstream CNN- and RNN-based baselines. This validates Mamba’s feasibility as a general-purpose, clinically promising EEG foundation model. To our knowledge, this is the first efficient, scalable SSM-based foundation model framework specifically designed for EEG, offering a novel pathway toward intelligent diagnosis of neurological disorders.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Deep Neural Architectures and Foundation Models
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

This work explores the potential of foundation models, specifically a Mamba-based selective state space model, for enhancing EEG analysis in neurological disorder diagnosis. EEG, crucial for diagnosing conditions like epilepsy, presents significant challenges due to its noisy, high-dimensional, and nonlinear nature. Traditional machine learning methods have made advances in automating EEG analysis but often fail to capture its complex spatio-temporal dynamics. Recent advances in deep learning, particularly in sequence modeling, offer new avenues for creating more generalized and expressive models capable of handling such complexities. By training a Mamba-based model on a large dataset containing seizure and non-seizure EEG recordings through a self-supervised reconstruction task followed by a seizure detection task, we demonstrate the model's effectiveness, achieving an AUROC of 0.72 on a held-out test set. This approach marks a significant step toward developing large-scale, clinically applicable foundation models for EEG data analysis.

Problem

• Enhancing EEG analysis for neurological disorder diagnosis
• Capturing complex spatio-temporal dynamics in noisy EEG data
• Developing clinically applicable foundation models for EEG

Innovation

• Mamba-based selective state space model
• Self-supervised reconstruction and seizure detection
• Handling EEG's noisy high-dimensional nonlinear nature