DeepThalamus: A novel deep learning method for automatic segmentation of brain thalamic nuclei from multimodal ultra-high resolution MRI

2024-01-15

arXiv.org

AI Summary

Automatic segmentation of thalamic nuclei in ultra-high-resolution (0.125 mm³) multimodal MRI (T1/T2/WMn) remains unsolved, as existing methods support only single-modality or standard-resolution (1 mm³) data. Method: We propose the first 3D deep learning framework tailored for ultra-high-resolution multimodal MRI segmentation. Built upon an enhanced 3D U-Net, it integrates cross-modal features and incorporates a semi-supervised training strategy to mitigate label scarcity. Concurrently, we construct the first voxel-wise, ultra-high-resolution thalamic nuclei annotation dataset. Results: Our method achieves accuracy on par with state-of-the-art models while demonstrating superior computational efficiency and robust generalization across diverse scanning protocols. We further provide a lightweight, T1-only adaptation for clinical feasibility. All code and end-to-end pipelines are publicly released, enabling plug-and-play deployment.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

The implication of the thalamus in multiple neurological pathologies makes it a structure of interest for volumetric analysis. In the present work, we have designed and implemented a multimodal volumetric deep neural network for the segmentation of thalamic nuclei at ultra-high resolution (0.125 mm3). Current tools either operate at standard resolution (1 mm3) or use monomodal data. To achieve the proposed objective, first, a database of semiautomatically segmented thalamic nuclei was created using ultra-high resolution T1, T2 and White Matter nulled (WMn) images. Then, a novel Deep learning based strategy was designed to obtain the automatic segmentations and trained to improve its robustness and accuaracy using a semisupervised approach. The proposed method was compared with a related state-of-the-art method showing competitive results both in terms of segmentation quality and efficiency. To make the proposed method fully available to the scientific community, a full pipeline able to work with monomodal standard resolution T1 images is also proposed.

Problem

• Automatic segmentation of thalamic nuclei from ultra-high resolution MRI.
• Improving robustness and accuracy using a deep learning approach.
• Providing a pipeline for monomodal standard resolution T1 images.

Innovation

• Multimodal volumetric deep neural network
• Ultra-high resolution MRI segmentation
• Semi-supervised learning for robustness