Multimodal Brain Tumour Classification Using Feature Fusion

2026-06-09

AI Summary

This study addresses the limitation of existing brain tumor classification methods that predominantly rely on single-modality imaging and fail to emulate the clinical diagnostic paradigm integrating both imaging and quantitative features. To bridge this gap, the authors propose a dual-branch deep network that processes MRI images and 91-dimensional radiomic features separately, incorporating a novel gated fusion mechanism and bidirectional cross-modal attention to enable effective multimodal representation synergy. Evaluated on a balanced dataset of 7,200 cases, the proposed approach substantially outperforms unimodal baselines, with the gated fusion variant achieving a classification accuracy of 96.13%. This work represents the first systematic effort to successfully integrate raw imaging data and high-dimensional radiomic features within an end-to-end deep learning framework.

Abstract

Clinicians diagnose brain tumors by synthesizing patient symptoms, medical history, and quantitative imaging data from modalities such as MRI and CT scans into a unified clinical judgement. However, most deep learning models rely on MRI/CT images alone, failing to replicate the clinicians multimodal reasoning. We explore a two-branch multimodal network combining raw MRI scans with 91 extracted radiomic features (intensity, texture, shape, and boundary descriptors) to classify brain tumors into glioma, meningioma, pituitary, and no-tumor. A pre-trained CNN backbone encodes the image stream, whereas a dedicated MLP encodes the radiomic stream. Both streams are fused via concatenation, gated, or bidirectional cross-modal attention strategies. Across nine experimental runs on a balanced 7,200 image dataset, all multimodal configurations outperform unimodal baselines with gated fusion achieving the best accuracy of 96.13%.

Problem

• multimodal
• brain tumor classification
• radiomic features
• feature fusion
• deep learning

Innovation

• multimodal fusion
• radiomic features
• gated fusion
• cross-modal attention
• brain tumor classification