Principled Feature Disentanglement for High-Fidelity Unified Brain MRI Synthesis

Unknown Date

AI Summary

To address inconsistent analysis in multi-sequence brain MRI clinical practice caused by frequent missing modalities, this paper proposes HF-GAN, a unified synthesis framework. Methodologically, it introduces (1) a principle-driven structured feature disentanglement mechanism that explicitly separates complementary anatomical information from modality-specific information; (2) a hybrid-flow architecture—combining multi-to-one and parallel one-to-one synthesis—integrated with a Channel-Attention-based Feature Fusion (CAFF) module for dynamic latent-space integration; and (3) a modality injector and 2D slice-level modeling strategy to enhance computational efficiency and generalizability. Evaluated on both healthy and pathological datasets, HF-GAN achieves state-of-the-art performance, significantly outperforming mainstream 3D synthesis methods. When applied to missing-modality imputation, it improves downstream brain tumor segmentation Dice scores by over 5%, demonstrating strong clinical utility.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Humans and AI: Brain-Sensing and Analysis

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Multisequence Magnetic Resonance Imaging (MRI) provides a more reliable diagnosis in clinical applications through complementary information across sequences. However, in practice, the absence of certain MR sequences is a common problem that can lead to inconsistent analysis results. In this work, we propose a novel unified framework for synthesizing multisequence MR images, called hybrid-fusion GAN (HF-GAN). The fundamental mechanism of this work is principled feature disentanglement, which aligns the design of the architecture with the complexity of the features. A powerful many-to-one stream is constructed for the extraction of complex complementary features, while utilizing parallel, one-to-one streams to process modality-specific information. These disentangled features are dynamically integrated into a common latent space by a channel attention-based fusion module (CAFF) and then transformed via a modality infuser to generate the target sequence. We validated our framework on public datasets of both healthy and pathological brain MRI. Quantitative and qualitative results show that HF-GAN achieves state-of-the-art performance, with our 2D slice-based framework notably outperforming a leading 3D volumetric model. Furthermore, the utilization of HF-GAN for data imputation substantially improves the performance of the downstream brain tumor segmentation task, demonstrating its clinical relevance.

Problem

• Synthesizing missing MRI sequences to ensure consistent analysis
• Disentangling complementary and modality-specific features for image generation
• Improving brain tumor segmentation through high-fidelity data imputation

Innovation

• Hybrid-fusion GAN framework for unified MRI synthesis
• Principled feature disentanglement with parallel processing streams
• Channel attention fusion module for dynamic feature integration