CoRe-BT: A Multimodal Radiology-Pathology-Text Benchmark for Robust Brain Tumor Typing

2026-03-03

AI Summary

Accurate subtyping of brain tumors relies on multimodal clinical data—including MRI, histopathology images, and textual reports—yet real-world diagnosis often encounters missing modalities. To address this challenge, this work introduces CoRe-BT, the first benchmark dataset for brain tumor analysis that explicitly supports joint modeling of radiology, pathology, and text while annotating modality-missing scenarios. CoRe-BT encompasses 310 patients, six glioma subtypes, and expert-annotated tumor region masks. Building upon this dataset, we propose a unified framework integrating multimodal fusion, cross-modal representation learning, region-aware modeling, and auxiliary tasks, enabling both MRI-only and multimodal inference. Baseline experiments demonstrate the complementary value of multimodal data and establish a new paradigm for robust tumor subtyping under incomplete clinical information.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Accurate brain tumor typing requires integrating heterogeneous clinical evidence, including magnetic resonance imaging (MRI), histopathology, and pathology reports, which are often incomplete at the time of diagnosis. We introduce CoRe-BT, a cross-modal radiology-pathology-text benchmark for brain tumor typing, designed to study robust multimodal learning under missing modality conditions. The dataset comprises 310 patients with multi-sequence brain MRI (T1, T1c, T2, FLAIR), including 95 cases with paired H&E-stained whole-slide pathology images and pathology reports. All cases are annotated with tumor type and grade, and MRI volumes include expert-annotated tumor masks, enabling both region-aware modeling and auxiliary learning tasks. Tumors are categorized into six clinically relevant classes capturing the heterogeneity of common and rare glioma subtypes. We evaluate tumor typing under variable modality availability by comparing MRI-only models with multimodal approaches that incorporate pathology information when present. Baseline experiments demonstrate the feasibility of multimodal fusion and highlight complementary modality contributions across clinically relevant typing tasks. CoRe-BT provides a grounded testbed for advancing multimodal glioma typing and representation learning in realistic scenarios with incomplete clinical data.

Problem

• brain tumor typing
• multimodal learning
• missing modality
• radiology-pathology integration
• clinical data incompleteness

Innovation

• multimodal learning
• missing modality robustness
• brain tumor typing
• radiology-pathology-text fusion
• glioma subtyping