Nexus-INR: Diverse Knowledge-guided Arbitrary-Scale Multimodal Medical Image Super-Resolution

2025-08-05

AI Summary

To address structural-modality feature coupling and insufficient cross-modal knowledge transfer in arbitrary-resolution super-resolution (ARSR) of multimodal medical images, this paper proposes Nexus-INR—a neural implicit representation framework. It employs a dual-branch encoder to disentangle anatomical structure features from modality-specific features and introduces a cross-modal attention-driven knowledge distillation mechanism. Furthermore, it jointly optimizes reconstruction and downstream tasks by integrating self-supervised consistency loss with a lightweight segmentation head. Compared to conventional CNN-based ARSR methods and existing INR approaches, Nexus-INR achieves significant improvements on BraTS2020: +1.23 dB in PSNR, +0.028 in SSIM for super-resolution, and +1.8% in Dice score for tumor segmentation. These results demonstrate its superior resolution adaptability, multimodal generalizability, and task synergy.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Arbitrary-resolution super-resolution (ARSR) provides crucial flexibility for medical image analysis by adapting to diverse spatial resolutions. However, traditional CNN-based methods are inherently ill-suited for ARSR, as they are typically designed for fixed upsampling factors. While INR-based methods overcome this limitation, they still struggle to effectively process and leverage multi-modal images with varying resolutions and details. In this paper, we propose Nexus-INR, a Diverse Knowledge-guided ARSR framework, which employs varied information and downstream tasks to achieve high-quality, adaptive-resolution medical image super-resolution. Specifically, Nexus-INR contains three key components. A dual-branch encoder with an auxiliary classification task to effectively disentangle shared anatomical structures and modality-specific features; a knowledge distillation module using cross-modal attention that guides low-resolution modality reconstruction with high-resolution reference, enhanced by self-supervised consistency loss; an integrated segmentation module that embeds anatomical semantics to improve both reconstruction quality and downstream segmentation performance. Experiments on the BraTS2020 dataset for both super-resolution and downstream segmentation demonstrate that Nexus-INR outperforms state-of-the-art methods across various metrics.

Problem

• Overcoming fixed upsampling limitations in medical image super-resolution
• Enhancing multi-modal image processing with varying resolutions
• Integrating anatomical semantics to improve reconstruction and segmentation

Innovation

• Dual-branch encoder disentangles shared and specific features
• Cross-modal attention guides reconstruction with high-resolution reference
• Integrated segmentation embeds semantics to enhance reconstruction