Prostate Cancer Classification Using Multimodal Feature Fusion and Explainable AI

2025-07-28

AI Summary

To address the challenges of insufficient multimodal data fusion and limited model interpretability in prostate cancer diagnosis—particularly for intermediate-risk cases—this paper proposes a BERT–Random Forest feature-level fusion framework: clinical text is encoded using a lightweight BERT variant, while laboratory test values are modeled via Random Forest; features from both modalities are concatenated to enable complementary representation learning, and SHAP is employed for both global and local interpretability analysis. Evaluated on the PLCO-NIH dataset, the method achieves 98% accuracy, 99% AUC, and 89% F1-score, with recall for Stage II and III cancers improved to 90%, significantly outperforming unimodal baselines. Key contributions include: (i) the first integration of a lightweight BERT architecture with tree-based models for multimodal prostate cancer diagnosis; (ii) empirical validation of strong complementarity between clinical text and numerical laboratory data; and (iii) simultaneous achievement of high predictive accuracy, low computational overhead, and clinically meaningful interpretability.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Computer Vision: Multi-modal Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Prostate cancer, the second most prevalent male malignancy, requires advanced diagnostic tools. We propose an explainable AI system combining BERT (for textual clinical notes) and Random Forest (for numerical lab data) through a novel multimodal fusion strategy, achieving superior classification performance on PLCO-NIH dataset (98% accuracy, 99% AUC). While multimodal fusion is established, our work demonstrates that a simple yet interpretable BERT+RF pipeline delivers clinically significant improvements - particularly for intermediate cancer stages (Class 2/3 recall: 0.900 combined vs 0.824 numerical/0.725 textual). SHAP analysis provides transparent feature importance rankings, while ablation studies prove textual features' complementary value. This accessible approach offers hospitals a balance of high performance (F1=89%), computational efficiency, and clinical interpretability - addressing critical needs in prostate cancer diagnostics.

Problem

• Improving prostate cancer classification using multimodal AI fusion
• Enhancing interpretability in cancer diagnostics with explainable AI
• Balancing performance and efficiency in clinical cancer detection

Innovation

• Multimodal fusion of BERT and Random Forest
• Explainable AI with SHAP feature analysis
• High accuracy prostate cancer classification