Temporal Fusion Nexus: A task-agnostic multi-modal embedding model for clinical narratives and irregular time series in post-kidney transplant care

2026-01-13

AI Summary

This study addresses the challenge of predicting critical clinical outcomes—graft loss, rejection, and mortality—following kidney transplantation by proposing the first end-to-end multimodal embedding model that jointly integrates irregularly sampled time-series data and unstructured clinical text. The approach leverages decoupled representation learning and irregular time-series modeling to produce interpretable, task-agnostic embeddings. Evaluated on a cohort of 3,382 patients, the model significantly outperforms existing unimodal and state-of-the-art multimodal methods, achieving AUCs of 0.96, 0.84, and 0.86 for graft loss, rejection, and mortality prediction, respectively. This work demonstrates, for the first time, effective synergy and cross-task generalization across heterogeneous data sources in kidney transplant care.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

We introduce Temporal Fusion Nexus (TFN), a multi-modal and task-agnostic embedding model to integrate irregular time series and unstructured clinical narratives. We analysed TFN in post-kidney transplant (KTx) care, with a retrospective cohort of 3382 patients, on three key outcomes: graft loss, graft rejection, and mortality. Compared to state-of-the-art model in post KTx care, TFN achieved higher performance for graft loss (AUC 0.96 vs. 0.94) and graft rejection (AUC 0.84 vs. 0.74). In mortality prediction, TFN yielded an AUC of 0.86. TFN outperformed unimodal baselines (approx 10% AUC improvement over time series only baseline, approx 5% AUC improvement over time series with static patient data). Integrating clinical text improved performance across all tasks. Disentanglement metrics confirmed robust and interpretable latent factors in the embedding space, and SHAP-based attributions confirmed alignment with clinical reasoning. TFN has potential application in clinical tasks beyond KTx, where heterogeneous data sources, irregular longitudinal data, and rich narrative documentation are available.

Problem

• multi-modal embedding
• irregular time series
• clinical narratives
• post-kidney transplant care
• outcome prediction

Innovation

• multi-modal embedding
• irregular time series
• clinical narratives
• task-agnostic model
• interpretable representation