🤖 AI Summary
This work addresses the scarcity of annotated data in 3D medical imaging and the limited applicability of existing 2D zero-shot anomaly detection methods by proposing CS3F, a training-free 3D zero-shot anomaly detection framework. CS3F leverages a frozen 2D vision transformer and employs multi-axis slice decomposition, neighboring slice feature pooling, and a coarse-to-fine voxel-level tokenization strategy to preserve lesion signals while enabling high-resolution anomaly localization. It introduces a novel cross-subject similarity metric to compute anomaly scores. Experiments demonstrate that CS3F achieves effective performance on brain MRI (metastases, gliomas, stroke) and lung CT datasets, validating that frozen 2D foundation models can be successfully adapted for 3D anomaly detection, with fine-grained tokenization efficacy influenced by lesion contrast and imaging modality.
📝 Abstract
Zero-shot anomaly detection (ZSAD) is attractive for medical imaging because clinical systems must handle heterogeneous acquisition protocols, changing patient populations, and pathologies for which annotated training data may be unavailable. Most existing zero-shot anomaly detection methods are designed for 2D images, and their direct extension to 3D medical volumes is limited by the scarcity of large-scale volumetric foundation models or by the difficulty of utilizing volumetric context. We propose CS3F, a training-free batch-based framework for ZSAD in 3D medical images using 2D foundation models. Each volume is decomposed along multiple anatomical axes and encoded slice-wise by a 2D vision transformer. These are then converted into localized volumetric tokens by pooling neighboring slice features. Anomaly scores are obtained from cross-subject mutual similarity: tokens that lack close analogues in other subjects are assigned higher anomaly scores. To reduce the attenuation of focal lesion signals caused by depth pooling, we introduce a coarse-to-fine tokenization strategy that enables fine-resolution volumetric scoring without exhaustive matching. CS3F is evaluated on brain MRI across metastases, glioma, and stroke, as well as validated on lung CT to test generalizability beyond atlas-aligned brain MRI. The results show that frozen 2D foundation models can support anomaly localization in 3D medical images, and that the benefit of fine tokenization depends strongly on lesion contrast and imaging modality.