🤖 AI Summary
This study systematically evaluates the effectiveness of test-time augmentation (TTA) in medical image classification and finds that, contrary to common assumptions, TTA often significantly degrades accuracy—by as much as 31.6 percentage points—across most scenarios, with only marginal gains observed in specific dermatological tasks (+1.6%). Leveraging the MedMNIST v2 benchmark, four model scales, and multiple standard TTA strategies, the authors identify the primary cause of performance degradation as a mismatch between training and test distribution, exacerbated by incompatible batch normalization statistics. Notably, the work demonstrates for the first time that intensity-based transformations consistently outperform geometric ones. These findings challenge the prevailing assumption of TTA’s universal efficacy and underscore the necessity of carefully selecting augmentation strategies in medical imaging applications.
📝 Abstract
Test-time augmentation (TTA)--aggregating predictions over multiple augmented copies of a test input--is widely assumed to improve classification accuracy, particularly in medical imaging where it is routinely deployed in production systems and competition solutions. We present a systematic empirical study challenging this assumption across three MedMNIST v2 benchmarks and four architectures spanning three orders of magnitude in parameter count (21K to 11M). Our principal finding is that TTA with standard augmentation pipelines consistently degrades accuracy relative to single-pass inference, with drops as severe as 31.6 percentage points for ResNet-18 on pathology images. This degradation affects all architectures, including convolutional models, and worsens with more augmented views. The sole exception is ResNet-18 on dermatology images, which gains a modest +1.6%. We identify the distribution shift between augmented and training-time inputs--amplified by batch normalization statistics mismatch--as the primary mechanism. Our ablation studies show that augmentation strategy matters critically: intensity-only augmentations preserve more performance than geometric transforms, and including the original unaugmented image partially mitigates but does not eliminate the accuracy drop. These findings serve as a cautionary note for practitioners: TTA should not be applied as a default post-hoc improvement but must be validated on the specific model-dataset combination.