Language-Enhanced Generative Modeling for PET Synthesis from MRI and Blood Biomarkers

2025-11-04

AI Summary

Alzheimer’s disease (AD) diagnosis heavily relies on costly and inaccessible amyloid-β positron emission tomography (Aβ-PET) imaging. To address this, we propose a large language model (LLM)-enhanced multimodal generative framework that synthesizes Aβ-PET images end-to-end from blood-based biomarkers (BBMs) and structural MRI. Our method innovatively leverages an LLM for prompt engineering and cross-modal alignment—marking the first application of language-guided medical image generation for AD diagnosis. The synthesized PET images achieve high structural fidelity (SSIM = 0.920) and accurately replicate regional Aβ deposition patterns (Pearson *r* = 0.955). An automated classifier trained solely on synthetic PET data attains an AUC of 0.78 and diagnostic accuracy of 80%, significantly outperforming conventional unimodal approaches. This work establishes a novel paradigm for low-cost, widely accessible, and precise AD diagnosis.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Computer Vision: Multi-modal Vision

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Background: Alzheimer's disease (AD) diagnosis heavily relies on amyloid-beta positron emission tomography (Abeta-PET), which is limited by high cost and limited accessibility. This study explores whether Abeta-PET spatial patterns can be predicted from blood-based biomarkers (BBMs) and MRI scans. Methods: We collected Abeta-PET images, T1-weighted MRI scans, and BBMs from 566 participants. A language-enhanced generative model, driven by a large language model (LLM) and multimodal information fusion, was developed to synthesize PET images. Synthesized images were evaluated for image quality, diagnostic consistency, and clinical applicability within a fully automated diagnostic pipeline. Findings: The synthetic PET images closely resemble real PET scans in both structural details (SSIM = 0.920 +/- 0.003) and regional patterns (Pearson's r = 0.955 +/- 0.007). Diagnostic outcomes using synthetic PET show high agreement with real PET-based diagnoses (accuracy = 0.80). Using synthetic PET, we developed a fully automatic AD diagnostic pipeline integrating PET synthesis and classification. The synthetic PET-based model (AUC = 0.78) outperforms T1-based (AUC = 0.68) and BBM-based (AUC = 0.73) models, while combining synthetic PET and BBMs further improved performance (AUC = 0.79). Ablation analysis supports the advantages of LLM integration and prompt engineering. Interpretation: Our language-enhanced generative model synthesizes realistic PET images, enhancing the utility of MRI and BBMs for Abeta spatial pattern assessment and improving the diagnostic workflow for Alzheimer's disease.

Problem

• Predicting amyloid-beta PET spatial patterns from blood biomarkers and MRI scans
• Overcoming high cost and limited accessibility of Abeta-PET for Alzheimer's diagnosis
• Developing automated diagnostic pipeline using synthesized PET images for clinical application

Innovation

• Language-enhanced generative model synthesizes PET from MRI
• Model integrates blood biomarkers and MRI via LLM fusion
• Automated diagnostic pipeline combines PET synthesis and classification