Toward explainable AI approaches for breast imaging: adapting foundation models to diverse populations

2025-11-21

AI Summary

This study addresses the limited cross-population and cross-modal generalizability of foundational models—specifically BiomedCLIP—in BI-RADS breast density classification. We propose a multimodal training framework with weighted contrastive learning, integrating synthetic 2D mammograms, digital mammography (DM), and digital breast tomosynthesis (DBT) images while mitigating class imbalance. Our key innovations include a modality-adaptive weighting mechanism that dynamically adjusts loss contributions per imaging modality and GradCAM-based interpretability analysis to enhance clinical trustworthiness. On an internal test set, the model achieves 74.0% accuracy and an AUC ≥ 0.84. External validation on the RSNA and EMBED datasets yields AUCs of 0.80–0.93, substantially outperforming baseline methods. These results demonstrate robust generalization across diverse populations and imaging modalities, underscoring the framework’s clinical applicability and reliability in real-world breast imaging workflows.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Foundation models hold promise for specialized medical imaging tasks, though their effectiveness in breast imaging remains underexplored. This study leverages BiomedCLIP as a foundation model to address challenges in model generalization. BiomedCLIP was adapted for automated BI-RADS breast density classification using multi-modality mammographic data (synthesized 2D images, digital mammography, and digital breast tomosynthesis). Using 96,995 images, we compared single-modality (s2D only) and multi-modality training approaches, addressing class imbalance through weighted contrastive learning. Both approaches achieved similar accuracy (multi-modality: 0.74, single-modality: 0.73), with the multi-modality model offering broader applicability across different imaging modalities and higher AUC values consistently above 0.84 across BI-RADS categories. External validation on the RSNA and EMBED datasets showed strong generalization capabilities (AUC range: 0.80-0.93). GradCAM visualizations confirmed consistent and clinically relevant attention patterns, highlighting the models interpretability and robustness. This research underscores the potential of foundation models for breast imaging applications, paving the way for future extensions for diagnostic tasks.

Problem

• Adapting foundation models for breast density classification across populations
• Addressing model generalization challenges in multi-modality mammographic imaging
• Developing explainable AI approaches for robust breast imaging applications

Innovation

• Adapted BiomedCLIP foundation model for breast imaging
• Used multi-modality mammographic data and contrastive learning
• Achieved strong generalization with interpretable GradCAM visualizations