Multimodal MRI-Ultrasound AI for Prostate Cancer Detection Outperforms Radiologist MRI Interpretation: A Multi-Center Study

2025-01-31

AI Summary

To address the low detection rate, poor spatial localization, and registration errors of clinically significant prostate cancer (CsPCa) when using MRI or TRUS independently—leading to missed diagnoses—this study proposes the first multi-center validated cross-modal MRI-TRUS joint modeling framework. Built upon a 3D U-Net architecture, it introduces three key innovations: (1) multi-scale feature fusion, (2) temporal modeling of TRUS image sequences, and (3) an MRI-TRUS co-attention mechanism, enabling lesion-level precise detection and localization. Evaluated on 1,700 test cases, the framework achieves 80% sensitivity and a lesion-level Dice score of 42%. Compared to radiologists, it significantly improves specificity (88% vs. 78%) and lesion Dice (38% vs. 33%). Notably, it is the first method to surpass both unimodal AI models and clinical experts on lesion-level metrics, establishing a new benchmark for cross-modal prostate cancer diagnosis.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Pre-biopsy magnetic resonance imaging (MRI) is increasingly used to target suspicious prostate lesions. This has led to artificial intelligence (AI) applications improving MRI-based detection of clinically significant prostate cancer (CsPCa). However, MRI-detected lesions must still be mapped to transrectal ultrasound (TRUS) images during biopsy, which results in missing CsPCa. This study systematically evaluates a multimodal AI framework integrating MRI and TRUS image sequences to enhance CsPCa identification. The study included 3110 patients from three cohorts across two institutions who underwent prostate biopsy. The proposed framework, based on the 3D UNet architecture, was evaluated on 1700 test cases, comparing performance to unimodal AI models that use either MRI or TRUS alone. Additionally, the proposed model was compared to radiologists in a cohort of 110 patients. The multimodal AI approach achieved superior sensitivity (80%) and Lesion Dice (42%) compared to unimodal MRI (73%, 30%) and TRUS models (49%, 27%). Compared to radiologists, the multimodal model showed higher specificity (88% vs. 78%) and Lesion Dice (38% vs. 33%), with equivalent sensitivity (79%). Our findings demonstrate the potential of multimodal AI to improve CsPCa lesion targeting during biopsy and treatment planning, surpassing current unimodal models and radiologists; ultimately improving outcomes for prostate cancer patients.

Problem

• Prostate Cancer Detection
• Magnetic Resonance Imaging (MRI)
• Transrectal Ultrasound (TRUS)

Innovation

• AI-enhanced multi-modal imaging
• 3D UNet technique
• prostate cancer detection