Segmentation-Assisted Brain MRI Synthesis with Cross-Image Multi-Contrast Feature Memory Bank Retrieval Augmentation

2026-06-06

AI Summary

This work addresses the challenge of missing modalities in multi-contrast brain MRI, which often arises due to scanning constraints or protocol variations and hinders high-fidelity synthesis—particularly within tumor regions—and effective exploitation of multi-contrast contextual information. To this end, the authors propose a synthesis-driven, segmentation-assisted closed-loop framework that leverages generative adversarial networks to synthesize missing contrasts from any available input modalities. Crucially, tumor segmentation masks are incorporated as semantic feedback to enhance lesion realism. The method further introduces a novel dual memory-bank retrieval mechanism: one bank stores tumor masks while the other captures cross-image multi-contrast features, jointly enabling dynamic enhancement of both local semantics and global stylistic consistency. Experiments on the BraTS2020 and UCSF-BMSR datasets demonstrate that the proposed approach significantly outperforms existing state-of-the-art methods.

Abstract

Multi-contrast brain MRI provide complementary soft-tissue characteristics that aid in the screening and diagnosis of diseases. However, limited scanning time, image corruption and various imaging protocols often result in incomplete multi-contrast images. While current approaches excel in image synthesis, they often struggle to synthesize critical tumor regions and exploit contextual information in multi-contrast brain MRI effectively. To address this issue, we propose a synthesis-centric, segmentation-assisted closed-loop framework with retrieval augmentation synthesis. Our method overall takes a generative adversarial architecture, which aims to synthesize missing contrasts from any combination of available ones with a single model. To explicitly capture tumor semantics and focus synthesis on tumor regions, we add an auxiliary segmentation branch that predicts tumor masks and feeds them back as semantic conditioning in synthesis branch, thereby learning tumor-aware representations in the model and improving synthesis fidelity. Furthermore, we propose a dual-bank retrieval augmentation strategy. It dynamically queries two external knowledge bases, namely a tumor masks memory bank for crucial tumor context and cross-image contrast feature memory bank for global style information, to augment synthesis. Verified on two public multi-contrast magnetic resonance brain datasets: BraTs2020 and UCSF-BMSR, the proposed method is effective in handling medical brain images synthesis tasks and shows superior performance compared to previous methods. Code is available at:https://github.com/iBizzard/SSCF.git

Problem

• brain MRI synthesis
• multi-contrast MRI
• tumor region synthesis
• incomplete MRI data
• contextual information exploitation

Innovation

• segmentation-assisted synthesis
• retrieval augmentation
• multi-contrast MRI
• tumor-aware representation
• memory bank