Predicting Cognition from fMRI:A Comparative Study of Graph, Transformer, and Kernel Models Across Task and Rest Conditions

2025-07-28

AI Summary

This study addresses the challenge of predicting individual cognitive abilities from fMRI data to uncover underlying neurobiological mechanisms and advance precision medicine and early detection of neuropsychiatric disorders. Methodologically, we systematically benchmark graph neural networks (GNNs), Transformer-GNNs, and kernel ridge regression (KRR) on both resting-state and task-based fMRI, and propose a multimodal GNN framework integrating structural connectivity (SC) and functional connectivity (FC). Results show that task-based fMRI yields significantly higher predictive accuracy than resting-state fMRI; the SC-FC–integrated GNN achieves superior and more stable performance across most metrics; Transformer-GNN excels in dynamic modeling for task-based data but underperforms in resting-state settings; notably, its accuracy is not statistically distinguishable from FC-only KRR. This work establishes a reproducible benchmark framework for multimodal brain imaging and provides mechanistic insights into connectome-based cognitive prediction.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Computer Vision: Multi-modal Vision

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Predicting cognition from neuroimaging data in healthy individuals offers insights into the neural mechanisms underlying cognitive abilities, with potential applications in precision medicine and early detection of neurological and psychiatric conditions. This study systematically benchmarked classical machine learning (Kernel Ridge Regression (KRR)) and advanced deep learning (DL) models (Graph Neural Networks (GNN) and Transformer-GNN (TGNN)) for cognitive prediction using Resting-state (RS), Working Memory, and Language task fMRI data from the Human Connectome Project Young Adult dataset. Our results, based on R2 scores, Pearson correlation coefficient, and mean absolute error, revealed that task-based fMRI, eliciting neural responses directly tied to cognition, outperformed RS fMRI in predicting cognitive behavior. Among the methods compared, a GNN combining structural connectivity (SC) and functional connectivity (FC) consistently achieved the highest performance across all fMRI modalities; however, its advantage over KRR using FC alone was not statistically significant. The TGNN, designed to model temporal dynamics with SC as a prior, performed competitively with FC-based approaches for task-fMRI but struggled with RS data, where its performance aligned with the lower-performing GNN that directly used fMRI time-series data as node features. These findings emphasize the importance of selecting appropriate model architectures and feature representations to fully leverage the spatial and temporal richness of neuroimaging data. This study highlights the potential of multimodal graph-aware DL models to combine SC and FC for cognitive prediction, as well as the promise of Transformer-based approaches for capturing temporal dynamics. By providing a comprehensive comparison of models, this work serves as a guide for advancing brain-behavior modeling using fMRI, SC and DL.

Problem

• Compare machine learning models for cognitive prediction from fMRI
• Evaluate performance of graph, transformer, and kernel models across tasks
• Assess impact of structural and functional connectivity on prediction accuracy

Innovation

• Combines structural and functional connectivity in GNN
• Uses Transformer-GNN for temporal dynamics modeling
• Benchmarks Kernel Ridge Regression against deep learning