Domain Adaptation-Enhanced Searchlight: Enabling classification of brain states from visual perception to mental imagery

2024-08-02

AI Summary

This study addresses the cross-domain brain-state decoding challenge—from visual perception to mental imagery—within brain-computer interfaces and cognitive neuroscience. We propose the first fMRI searchlight analysis framework integrating domain adaptation (CORAL, DANN, MMD) to achieve robust cross-subject and cross-task (perception → imagery) mental imagery decoding. Critically, we discover—for the first time—that non-visual cortices, particularly the prefrontal and parietal regions, exhibit significant decodability during mental imagery. On our newly collected dataset of 18 participants, the method achieves statistically significant improvements in binary classification accuracy. It also demonstrates enhanced generalization on public multi-class fMRI datasets. Searchlight decoding precisely identifies a distributed network of high-decodability regions spanning occipital, parietal, and prefrontal cortices. All code and data are publicly released to support reproducibility and further research.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Computer Vision: Multi-modal Vision
• Humans and AI: Brain-Sensing and Analysis

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• User Modeling, Personalization and Recommendation: ML for personalized search and recommendations
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

In cognitive neuroscience and brain-computer interface research, accurately predicting imagined stimuli is crucial. This study investigates the effectiveness of Domain Adaptation (DA) in enhancing imagery prediction using primarily visual data from fMRI scans of 18 subjects. Initially, we train a baseline model on visual stimuli to predict imagined stimuli, utilizing data from 14 brain regions. We then develop several models to improve imagery prediction, comparing different DA methods. Our results demonstrate that DA significantly enhances imagery prediction in binary classification on our dataset, as well as in multiclass classification on a publicly available dataset. We then conduct a DA-enhanced searchlight analysis, followed by permutation-based statistical tests to identify brain regions where imagery decoding is consistently above chance across subjects. Our DA-enhanced searchlight predicts imagery contents in a highly distributed set of brain regions, including the visual cortex and the frontoparietal cortex, thereby outperforming standard cross-domain classification methods. The complete code and data for this paper have been made openly available for the use of the scientific community.

Problem

• Brain-Computer Interface
• Neural Prediction
• Cognitive State Recognition

Innovation

• Domain Adaptation
• Brain Activity Recognition
• Machine Learning in Neuroimaging