Foundation Models in Radiology: What, How, When, Why and Why Not

2024-11-27

Radiology

AI Summary

This paper addresses the absence of a unified paradigm and clinical integration pathway for radiology foundation models. Methodologically, it systematically constructs the first comprehensive framework for radiology foundation models by integrating self-supervised learning, multimodal pretraining, medical image–text alignment, domain adaptation, and interpretability evaluation, while explicitly formalizing synergistic mechanisms for clinical adaptability, data privacy, regulatory compliance, and secure deployment. Key contributions include: (1) the first standardized taxonomy and delineation of capability boundaries for radiology foundation models; (2) development of training and evaluation guidelines that jointly ensure technical robustness and clinical safety; and (3) establishment of a closed-loop pipeline spanning algorithmic development to clinical implementation. The framework advances patient care quality, augments radiologists and clinical decision-makers, and promotes responsible AI adoption in radiology.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Multi-modal Vision
• Cognitive Modeling & Cognitive Systems: Other Foundations of Cognitive Modeling & Systems

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Security and Privacy: Security and privacy of machine learning and AI applications
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Recent advances in artificial intelligence have witnessed the emergence of large-scale deep learning models capable of interpreting and generating both textual and imaging data. Such models, typically referred to as foundation models (FMs), are trained on extensive corpora of unlabeled data and demonstrate high performance across various tasks. FMs have recently received extensive attention from academic, industry, and regulatory bodies. Given the potentially transformative impact that FMs can have on the field of radiology, radiologists must be aware of potential pathways to train these radiology-specific FMs, including understanding both the benefits and challenges. Thus, this review aims to explain the fundamental concepts and terms of FMs in radiology, with a specific focus on the requirements of training data, model training paradigms, model capabilities, and evaluation strategies. Overall, the goal of this review is to unify technical advances and clinical needs for safe and responsible training of FMs in radiology to ultimately benefit patients, providers, and radiologists.

Problem

• Establish standardized terminology for radiology foundation models
• Facilitate training of radiology-specific foundation models
• Unify technical advances with clinical needs in radiology

Innovation

• Foundation models interpret radiology data
• Training on extensive unlabeled datasets
• Focus on clinical needs and safety