Distributional Drift Detection in Medical Imaging with Sketching and Fine-Tuned Transformer

2024-08-15

AI Summary

To address diagnostic performance degradation in medical imaging (CT and mammography) caused by post-deployment data distribution shift, this paper proposes a lightweight, robust real-time drift detection method. Our approach constructs a dynamic data sketch reference library using Count-Min Sketch and employs fine-tuned Vision Transformers (ViTs) for feature extraction, followed by cosine similarity–based drift discrimination. We innovatively integrate probabilistic data sketching with ViT fine-tuning—first applied to medical image drift detection. Additionally, we design an evaluation protocol robust to illumination variations and highly sensitive, capable of detecting as little as 1% salt-and-pepper or speckle noise. On mammography data, our method achieves 99.11% detection accuracy; cosine similarity discriminability between similar datasets improves from ≈50% to 99.1%, significantly outperforming existing approaches.

Technology Category

• Computer Vision: Medical and Biological Imaging
• Machine Learning: Calibration & Uncertainty Quantification
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

Distributional drift detection is important in medical applications as it helps ensure the accuracy and reliability of models by identifying changes in the underlying data distribution that could affect the prediction results of machine learning models. However, current methods have limitations in detecting drift, for example, the inclusion of abnormal datasets can lead to unfair comparisons. This paper presents an accurate and sensitive approach to detect distributional drift in CT-scan medical images by leveraging data-sketching and fine-tuning techniques. We developed a robust baseline library model for real-time anomaly detection, allowing for efficient comparison of incoming images and identification of anomalies. Additionally, we fine-tuned a pre-trained Vision Transformer model to extract relevant features, using mammography as a case study, significantly enhancing model accuracy to 99.11%. Combining with data-sketches and fine-tuning, our feature extraction evaluation demonstrated that cosine similarity scores between similar datasets provide greater improvements, from around 50% increased to 99.1%. Finally, the sensitivity evaluation shows that our solutions are highly sensitive to even 1% salt-and-pepper and speckle noise, and it is not sensitive to lighting noise (e.g., lighting conditions have no impact on data drift). The proposed methods offer a scalable and reliable solution for maintaining the accuracy of diagnostic models in dynamic clinical environments.

Problem

• Detecting distributional drift in CT-scan medical images
• Improving drift detection accuracy with sketching and fine-tuning
• Ensuring model reliability in dynamic clinical environments

Innovation

• Data-sketching for efficient anomaly detection
• Fine-tuned Vision Transformer for feature extraction
• Cosine similarity scores for improved accuracy