Graph Neural Networks with Transformer Fusion of Brain Connectivity Dynamics and Tabular Data for Forecasting Future Tobacco Use

2025-12-28

AI Summary

Integrating non-Euclidean dynamic functional connectivity graphs with Euclidean clinical tabular data in longitudinal neuroimaging studies remains challenging, particularly due to weak temporal modeling and structural heterogeneity. Method: We propose GNN-TF—a temporally aware graph neural network–Transformer fusion model—that introduces the first end-to-end architecture jointly modeling topological structure and temporal evolution across multimodal heterogeneous data. It integrates dynamic graph convolution, temporal positional encoding, and cross-modal feature alignment and fusion mechanisms. Results: Evaluated on the NCANDA cohort for predicting adolescent tobacco use within 12 months, GNN-TF achieves an AUC improvement of 8.2% over baselines including XGBoost, LSTM, GCN, and ST-GNN, significantly enhancing early-risk detection. This work establishes a novel, interpretable, and temporally sensitive paradigm for multimodal longitudinal neuroimaging prediction.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Integrating non-Euclidean brain imaging data with Euclidean tabular data, such as clinical and demographic information, poses a substantial challenge for medical imaging analysis, particularly in forecasting future outcomes. While machine learning and deep learning techniques have been applied successfully to cross-sectional classification and prediction tasks, effectively forecasting outcomes in longitudinal imaging studies remains challenging. To address this challenge, we introduce a time-aware graph neural network model with transformer fusion (GNN-TF). This model flexibly integrates both tabular data and dynamic brain connectivity data, leveraging the temporal order of these variables within a coherent framework. By incorporating non-Euclidean and Euclidean sources of information from a longitudinal resting-state fMRI dataset from the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA), the GNN-TF enables a comprehensive analysis that captures critical aspects of longitudinal imaging data. Comparative analyses against a variety of established machine learning and deep learning models demonstrate that GNN-TF outperforms these state-of-the-art methods, delivering superior predictive accuracy for predicting future tobacco usage. The end-to-end, time-aware transformer fusion structure of the proposed GNN-TF model successfully integrates multiple data modalities and leverages temporal dynamics, making it a valuable analytic tool for functional brain imaging studies focused on clinical outcome prediction.

Problem

• Integrates brain connectivity and tabular data for forecasting
• Addresses longitudinal imaging outcome prediction challenges
• Predicts future tobacco use with superior accuracy

Innovation

• GNN-TF integrates brain connectivity and tabular data
• Transformer fusion leverages temporal dynamics in framework
• Model outperforms existing methods in predictive accuracy