Spiking Neural Networks for fMRI-Based Visual Semantic Decoding

2026-07-21

AI Summary

This study addresses the limitations of current fMRI-based visual semantic decoding, which predominantly relies on features extracted from artificial neural networks (ANNs) whose alignment with brain activity remains questionable. For the first time, the authors systematically investigate the use of features derived from spiking neural networks (SNNs) as regression targets, employing a unified L2-regularized linear decoder to evaluate four SNN variants—each embodying distinct spiking dynamics—against ANN baselines on the GoD dataset. The results demonstrate that SNN-derived features substantially enhance brain decodability: prediction error drops from 0.7707 to 0.0282, and top-1 semantic accuracy improves from 0.1800 to 0.4400. These findings highlight the critical role of spiking dynamics and the number of temporal simulation steps in visual semantic brain decoding.

Abstract

Functional magnetic resonance imaging (fMRI)-based visual decoding aims to recover visual information from measured brain activity, commonly by mapping fMRI responses into latent visual features for downstream decoding tasks. Most existing methods learn mappings from fMRI responses to visual features extracted by artificial neural networks (ANNs), yet it remains unclear whether ANN-derived features provide suitable targets for brain decoding. In this study, we investigate spiking neural network (SNN)-derived visual features as alternative targets for fMRI-based visual decoding. We compare an ANN baseline with four SNN variants from the same architectural family, which differ in their spiking dynamics. To isolate the effect of the target features, all models use the same L2-regularized linear fMRI-to-feature decoder, while only the feature vectors used as regression targets are varied. Compared with the ANN baseline, SNN-derived features exhibit stronger alignment with fMRI responses and improve visual semantic decoding performance. For instance, on the GoD dataset, SNN-derived features reduce feature-prediction error from 0.7707 to 0.0282 and improve top-1 semantic decoding accuracy from 0.1800 to 0.4400. Ablation results further indicate that both spiking neural dynamics and temporal simulation steps contribute to the observed advantage. These findings support SNN-derived features as effective brain-decodable visual representations and highlight target feature design as an important component of fMRI-based visual decoding.

Problem

• fMRI-based visual decoding
• visual semantic decoding
• spiking neural networks
• brain-decodable representations
• target feature design

Innovation

• Spiking Neural Networks
• fMRI-based visual decoding
• brain-decodable representations
• spiking dynamics
• semantic decoding