Big, Bright, or Invisible: A Frozen-Feature Benchmark of 3D CT Foundation Models

2026-08-06

AI Summary

This study systematically evaluates the generalization capabilities of ten frozen 3D CT foundation models across multi-cohort diagnostic tasks in chest CT imaging, with a focus on their performance in detecting lesions of varying sizes and contrast levels. Through k-nearest neighbors, zero-shot prompting, linear probing, and cross-dataset validation, the authors find that model performance is primarily constrained by lesion physical characteristics—specifically spatial scale and contrast—rather than architectural design. Global pooling embeddings prove inadequate for effectively representing small, low-contrast lesions. Models incorporating fine-grained image tokens and vision–language alignment achieve overall superior performance, though lightweight supervised encoders remain competitive. The findings reveal that current foundation models reliably detect large or high-contrast abnormalities, while small, low-contrast lesions continue to pose a widespread challenge.

Abstract

Routine CT interpretation is inherently comprehensive, capturing incidental findings across the entire scan volume. 3D CT foundation models could assist this process by providing generalizable representations of anatomy and pathology. To evaluate their diagnostic breadth, we benchmark ten frozen CT encoders across three cohorts of thoracic CT scans, including an unseen internal clinical dataset, using $k$-nearest neighbors, zero-shot prompting, and linear probing. We find no universal state-of-the-art, with rankings fluctuating significantly depending on the evaluation context. While models combining fine-grained image tokenization with vision-language alignment generally perform best, a lightweight supervised encoder remains highly competitive, demonstrating that explicit labels can effectively substitute for scale. Crucially, rather than model architecture, we observe that the primary determinant of performance is a physical bottleneck: a finding's detectability scales with its contrast against surrounding tissue and its spatial extent. Through controlled within-organ comparisons, we empirically demonstrate that widespread or high-contrast abnormalities, such as devices and effusions, are reliably recovered. Conversely, small, low-contrast focal lesions remain a persistent challenge across all evaluated encoders. We attribute this to the inherent limitations of globally pooled embeddings, suggesting that accurately representing small, low-contrast structures will require region- or lesion-level pretraining.

Problem

• 3D CT foundation models
• diagnostic breadth
• small low-contrast lesions
• frozen-feature benchmark
• anomaly detectability

Innovation

• frozen-feature benchmark
• 3D CT foundation models
• diagnostic breadth
• global pooling limitation
• lesion-level pretraining