CodeBrain: Impute Any Brain MRI via Instance-specific Scalar-quantized Codes

2025-01-30

AI Summary

This paper addresses the unified missing-modality imputation problem in multimodal brain MRI. We propose a cross-modal reconstruction method based on instance-level scalar quantized code prediction. Our key contributions are: (1) a modality-agnostic, instance-adaptive shared quantized latent space with a cross-modal feature compensation mechanism; (2) a two-stage training paradigm—first jointly optimizing latent space learning and cross-modal feature alignment, then supervising full-modality code prediction via a customized hierarchical loss; and (3) a dual-encoder architecture coupled with self-supervised reconstruction. Evaluated on IXI and BraTS 2023, our method significantly outperforms four state-of-the-art baselines. It is the first to achieve high-fidelity, robust cross-modal imputation from arbitrary single- or multi-modality inputs, establishing new SOTA performance for this task.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

MRI imputation aims to synthesize the missing modality from one or more available ones, which is highly desirable since it reduces scanning costs and delivers comprehensive MRI information to enhance clinical diagnosis. In this paper, we propose a unified model, CodeBrain, designed to adapt to various brain MRI imputation scenarios. The core design lies in casting various inter-modality transformations as a full-modality code prediction task. To this end, CodeBrain is trained in two stages: Reconstruction and Code Prediction. First, in the Reconstruction stage, we reconstruct each MRI modality, which is mapped into a shared latent space followed by a scalar quantization. Since such quantization is lossy and the code is low dimensional, another MRI modality belonging to the same subject is randomly selected to generate common features to supplement the code and boost the target reconstruction. In the second stage, we train another encoder by a customized grading loss to predict the full-modality codes from randomly masked MRI samples, supervised by the corresponding quantized codes generated from the first stage. In this way, the inter-modality transformation is achieved by mapping the instance-specific codes in a finite scalar space. We evaluated the proposed CodeBrain model on two public brain MRI datasets (i.e., IXI and BraTS 2023). Extensive experiments demonstrate that our CodeBrain model achieves superior imputation performance compared to four existing methods, establishing a new state of the art for unified brain MRI imputation. Codes will be released.

Problem

• MRI Image Completion
• Data Reconstruction
• Medical Diagnosis

Innovation

• CodeBrain
• MRI Reconstruction
• Coding Prediction