Reduce, Then Encode: Multiscale Volumetric Reduction for 2D Foundation Models in Brain MRI

2026-09-28

AI Summary

This study addresses the dimensional mismatch that hinders the direct application of 2D foundation models to 3D brain MRI. We propose a multi-scale volumetric dimensionality reduction adaptation method employing a “reduce-then-encode” strategy. Using unlabeled data, fixed projection matrices are constructed via decentralized PCA and multi-scale spatial descriptors to compress 3D volumes into a compact set of 2D components. These components are subsequently fed into a frozen 2D foundation model for feature encoding and linear probing. Notably, the entire adaptation process requires no gradient-based optimization. Experiments demonstrate that our approach significantly outperforms existing 2D-to-3D adaptation schemes and baseline models on datasets such as ADNI, while exhibiting strong cross-domain generalization capabilities.

Abstract

Pretrained 2D foundation models offer a practical alternative to dedicated 3D pretraining for brain structural magnetic resonance imaging (sMRI), but their use on volumetric data requires bridging the mismatch between a 2D encoder and a 3D volume input. Existing methods typically encode slices independently and integrate their features afterwards. We introduce Multiscale Volumetric Reduction (MVR), a reduce-then-encode approach that compresses each anatomical view from (D) slices into (M<<D) complementary 2D components before foundation-model encoding. MVR combines an uncentered-PCA base component derived from the original through-plane intensities with residual detail components constructed from multiscale spatial descriptors. The reduction is estimated from the training volumes without diagnostic labels or gradient-based optimization and remains fixed thereafter. The resulting components are independently processed by a shared frozen 2D foundation model and concatenated for linear probing. Under this frozen-encoder setting, MVR achieves strong overall performance across ADNI, OASIS, and ABIDE relative to the evaluated 2D-to-3D adaptation methods and simple input-reduction baselines, while also generalizing strongly from ADNI to AIBL.

Problem

• 2D foundation models
• brain MRI
• volumetric data
• 2D-to-3D adaptation
• structural MRI

Innovation

• Multiscale Volumetric Reduction
• 2D Foundation Models
• Brain MRI
• Reduce-then-Encode
• Uncentered-PCA