A Plug-and-Play Method for Guided Multi-contrast MRI Reconstruction based on Content/Style Modeling

2024-09-20

arXiv.org

AI Summary

To address the challenge of undersampled multi-contrast MRI reconstruction without paired training data, this paper proposes PnP-MUNIT: a two-stage, interpretable plug-and-play guided reconstruction method. It employs a MUNIT-based architecture to explicitly decouple anatomical content from contrast-specific style, leveraging high-quality reference contrast images as structural priors to guide reconstruction of undersampled target contrasts. Integrated within a plug-and-play optimization framework, it enforces data consistency and models multi-coil k-space acquisitions, enabling iterative refinement with guaranteed convergence. Its key innovation lies in establishing the first clinically interpretable content/style-decoupled guidance paradigm for cross-contrast knowledge transfer—without requiring paired data. On the NYU fastMRI and an internal clinical dataset, PnP-MUNIT achieves a 32.6% higher acceleration factor at equivalent SSIM compared to state-of-the-art methods; radiologist diagnostic quality assessments further demonstrate a 33.3% speedup over conventional clinical reconstructions.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Since multiple MRI contrasts of the same anatomy contain redundant information, one contrast can be used as a prior for guiding the reconstruction of an undersampled subsequent contrast. To this end, several learning-based guided reconstruction methods have been proposed. However, a key challenge is the requirement of large paired training datasets comprising raw data and aligned reference images. We propose a modular two-stage approach for guided reconstruction addressing this issue, which additionally provides an explanatory framework for the multi-contrast problem in terms of the shared and non-shared generative factors underlying two given contrasts. A content/style model of two-contrast image data is learned from a largely unpaired image-domain dataset and is subsequently applied as a plug-and-play operator in iterative reconstruction. The disentanglement of content and style allows explicit representation of contrast-independent and contrast-specific factors. Based on this, incorporating prior information into the reconstruction reduces to simply replacing the aliased content of the image estimate with high-quality content derived from the reference scan. Combining this component with a data consistency step and introducing a general corrective process for the content yields an iterative scheme. We name this novel approach PnP-MUNIT. Various aspects like interpretability and convergence are explored via simulations. Furthermore, its practicality is demonstrated on the NYU fastMRI DICOM dataset and two in-house multi-coil raw datasets, obtaining up to 32.6% more acceleration over learning-based non-guided reconstruction for a given SSIM. In a radiological task, PnP-MUNIT allowed 33.3% more acceleration over clinical reconstruction at diagnostic quality.

Problem

• Reduces need for large paired MRI datasets for reconstruction.
• Proposes modular approach for guided multi-contrast MRI reconstruction.
• Enhances MRI reconstruction speed without losing diagnostic quality.

Innovation

• Modular two-stage guided MRI reconstruction approach
• Content/style model from unpaired image datasets
• Plug-and-play iterative reconstruction with prior information