DITTO: A Context-aware Pickle-based Pre-Trained Model Scanner for Effective Security Audits

2026-10-07

AI Summary

This study addresses the deserialization attack risks inherent in Pickle-based pretrained models and the coverage-precision imbalance of existing scanners. We propose the first stack-based, context-aware security scanner that traces Pickle virtual machine state transitions and incorporates contextual semantic analysis to infer genuine model intent for precise auditing. Additionally, we construct PickleBench, a comprehensive benchmark encompassing extension registration attacks. Experimental results demonstrate that our approach achieves 100% coverage with zero false negatives, a mere 0.7% false positive rate, and an F1 score of 0.966, significantly outperforming state-of-the-art techniques.

Abstract

Pre-trained models (PTMs) are widely distributed as serialized binaries, but their reuse often exposes software supply chains to deserialization attacks. Despite the emergence of safer serialization formats, the unsafe Pickle format remains prevalent: our analysis of over 10,000 popular Hugging Face repositories reveals that 9.3% rely on Pickle. While many defense mechanisms have been proposed, state-of-the-art model scanners suffer from a coverage-precision gap, missing security-sensitive behaviors and generating excessive false alerts. In this paper, we introduce DITTO, the first stack-based, context-aware scanner for Pickle-based PTMs. DITTO faithfully tracks Pickle virtual machine state transitions and performs context-aware semantic analysis to infer model intentions. We also present PickleBench, a benchmark of 959 benign and 92 malicious real-world models, including extension registry attacks previously missed by existing tools. Across multiple evaluations, DITTO achieves 100% scanning coverage, a 0% false-negative rate, and a 0.7% false-positive rate, yielding an F1 score of 0.966, significantly outperforming state-of-the-art scanners. By minimizing false alerts while preserving detection accuracy, DITTO generates actionable security reports with contextual evidence, enabling safe PTM reuse and strengthening software supply chain integrity.

Problem

• Pre-trained models
• Pickle deserialization
• Security auditing
• Software supply chain
• Model scanner

Innovation

• Context-aware Scanner
• Pickle Virtual Machine
• Pre-trained Model Security
• Semantic Analysis
• Software Supply Chain