Augmented Intelligence for Multimodal Virtual Biopsy in Breast Cancer Using Generative Artificial Intelligence

2025-01-31

AI Summary

Full-field digital mammography (FFDM) exhibits limited diagnostic performance for dense or fibrocystic breasts; contrast-enhanced spectral mammography (CESM) faces barriers to clinical adoption due to ionizing radiation, iodinated contrast agent requirements, and low accessibility; and biopsy remains overly relied upon. Method: We propose a multimodal virtual biopsy framework featuring: (i) a novel conditional generative adversarial network (cGAN) for synthesizing CESM images directly from FFDM inputs—enabling non-invasive, low-dose multimodal integration; (ii) the first publicly available paired FFDM–CESM dataset; and (iii) a dual-path CNN with cross-modal feature fusion. Results: Synthesized CESM significantly improves benign–malignant classification (AUC = 0.947), outperforming single-modality FFDM (AUC = 0.862). Crucially, the framework maintains high sensitivity and specificity even in CESM-absent clinical settings.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Full-Field Digital Mammography (FFDM) is the primary imaging modality for routine breast cancer screening; however, its effectiveness is limited in patients with dense breast tissue or fibrocystic conditions. Contrast-Enhanced Spectral Mammography (CESM), a second-level imaging technique, offers enhanced accuracy in tumor detection. Nonetheless, its application is restricted due to higher radiation exposure, the use of contrast agents, and limited accessibility. As a result, CESM is typically reserved for select cases, leaving many patients to rely solely on FFDM despite the superior diagnostic performance of CESM. While biopsy remains the gold standard for definitive diagnosis, it is an invasive procedure that can cause discomfort for patients. We introduce a multimodal, multi-view deep learning approach for virtual biopsy, integrating FFDM and CESM modalities in craniocaudal and mediolateral oblique views to classify lesions as malignant or benign. To address the challenge of missing CESM data, we leverage generative artificial intelligence to impute CESM images from FFDM scans. Experimental results demonstrate that incorporating the CESM modality is crucial to enhance the performance of virtual biopsy. When real CESM data is missing, synthetic CESM images proved effective, outperforming the use of FFDM alone, particularly in multimodal configurations that combine FFDM and CESM modalities. The proposed approach has the potential to improve diagnostic workflows, providing clinicians with augmented intelligence tools to improve diagnostic accuracy and patient care. Additionally, as a contribution to the research community, we publicly release the dataset used in our experiments, facilitating further advancements in this field.

Problem

• Breast Cancer Detection
• Contrast-Enhanced Spectral Mammography (CESM)
• Non-invasive Biopsy Reduction

Innovation

• Deep Learning
• Breast Cancer Diagnosis
• FFDM and CESM Integration