Here Comes the Explanation: A Shapley Perspective on Multi-contrast Medical Image Segmentation

2025-04-06

AI Summary

Existing pixel-level interpretability methods for multi-contrast MRI tumor segmentation lack clinical interpretability and fail to characterize inter-contrast interaction mechanisms. Method: We propose the first contrast-level Shapley value attribution framework, attributing model decisions to MRI contrasts (e.g., T1, T2, FLAIR, T1c) rather than individual pixels, enabling clinically meaningful analysis of contrast fusion behavior. Results: Quantitative evaluation on the BraTS benchmark reveals that U-Net exhibits strong bias toward T1 and FLAIR contrasts, whereas Swin-UNETR achieves balanced, cross-contrast contributions—first systematically uncovering inherent architectural biases and compensatory fusion mechanisms in mainstream models. This work bridges a critical theoretical gap in explainable AI for multi-contrast medical imaging, substantially enhancing clinicians’ trust in model rationales and facilitating clinical adoption.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Calibration & Uncertainty Quantification
• Humans and AI: Explainable AI (XAI) for Human Understanding

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Deep learning has been successfully applied to medical image segmentation, enabling accurate identification of regions of interest such as organs and lesions. This approach works effectively across diverse datasets, including those with single-image contrast, multi-contrast, and multimodal imaging data. To improve human understanding of these black-box models, there is a growing need for Explainable AI (XAI) techniques for model transparency and accountability. Previous research has primarily focused on post hoc pixel-level explanations, using methods gradient-based and perturbation-based apporaches. These methods rely on gradients or perturbations to explain model predictions. However, these pixel-level explanations often struggle with the complexity inherent in multi-contrast magnetic resonance imaging (MRI) segmentation tasks, and the sparsely distributed explanations have limited clinical relevance. In this study, we propose using contrast-level Shapley values to explain state-of-the-art models trained on standard metrics used in brain tumor segmentation. Our results demonstrate that Shapley analysis provides valuable insights into different models' behavior used for tumor segmentation. We demonstrated a bias for U-Net towards over-weighing T1-contrast and FLAIR, while Swin-UNETR provided a cross-contrast understanding with balanced Shapley distribution.

Problem

• Explain multi-contrast MRI segmentation using Shapley values
• Address limitations of pixel-level explanations in medical imaging
• Evaluate model biases in brain tumor segmentation tasks

Innovation

• Uses contrast-level Shapley values for explanations
• Explains multi-contrast MRI segmentation models
• Compares U-Net and Swin-UNETR contrast biases