NeuroCBIR: A Fast and Accurate Image Retrieval System for Whole-Brain and Region-Specific MRI

2026-10-05

AI Summary

This study addresses the limitations of existing neuroimaging retrieval methods, which are often constrained by small sample sizes, single-region focus, and coarse-grained labels. We propose a framework enabling rapid whole-brain and region-level 3D MRI retrieval. Methodologically, it integrates variational autoencoders with contrastive learning to extract features across 103 brain regions, generating scan-level embeddings and constructing a large-scale precomputed index for flexible querying. The framework achieves zero-shot age prediction and multi-class pathology stratification, with re-identification mAP@5 exceeding 98.4%. Embedding extraction requires only 18.7 seconds per scan, while retrieval is near-instantaneous. By open-sourcing over 26,000 precomputed embeddings, this work delivers high retrieval accuracy, computational efficiency, and clinical generalizability.

Abstract

Content-based image retrieval (CBIR) in neuroimaging enables the identification of structurally similar brain scans, supporting diagnosis, prognosis, and treatment planning; however, existing methods are often limited to small datasets, single brain regions, or coarse class labels, thereby restricting their clinical utility and generalizability. Here, we present NeuroCBIR, a framework for fast and flexible retrieval of both whole-brain and region-specific 3D T1w MRI scans. A total of 103 cortical and subcortical regions are extracted to enable both whole-brain and region-level queries. NeuroCBIR leverages latent representations learned by a variational autoencoder (VAE) combined with contrastive learning, producing scan-specific embeddings that capture anatomical patterns. These embeddings were evaluated for subject re-identification, zero-shot age prediction, and zero-shot multi-class pathology stratification. Re-identification performance was high across both whole-brain and brain-region levels (mean average precision across the top-5 retrieved images (mAP@5)>= 98.4%), with robust generalization across datasets and acquisition conditions. While NeuroCBIR is not trained for age prediction or pathology stratification, zero-shot evaluations for these two tasks demonstrate that the embeddings encode meaningful information for downstream tasks. Embedding extraction on a 4-core CPU required approximately 18.7 s per scan, whereas similarity search was effectively instantaneous (less than 0.01 s). NeuroCBIR is publicly available for brain MRI with more than 26,000 precomputed T1w MRI embeddings. It supports reproducible research, region-specific flexibility, and clinically meaningful personalized diagnostic support. The software is available at https://github.com/minnelab/NeuroCBIR.

Problem

• Neuroimaging
• Content-based image retrieval
• Brain MRI
• Clinical utility
• Generalizability

Innovation

• Content-based image retrieval
• Variational autoencoder
• Contrastive learning
• Zero-shot generalization
• Brain MRI