DyePack: Provably Flagging Test Set Contamination in LLMs Using Backdoors

2025-05-29

AI Summary

This work addresses the challenge of detecting test-set contamination in large language model (LLM) evaluation without access to model internals. We propose the first black-box verification framework that requires no model querying. Methodologically, we introduce a novel, provably low-false-positive-rate detection mechanism based on multi-random-target backdoors—integrating backdoor attacks, statistical hypothesis testing, and randomized trigger–target mapping, enabling theoretically grounded, exact false positive rate calculation. On MMLU-Pro and BBH benchmarks, our method achieves a false positive rate of 0.000017% while attaining 100% contamination detection; on the Alpaca open-generation task, it identifies all contaminated instances with only a 0.127% false positive rate. To our knowledge, this is the first black-box approach that simultaneously ensures high sensitivity and rigorously bounded false positives, establishing a theoretically sound and practically deployable auditing paradigm for trustworthy LLM evaluation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Open benchmarks are essential for evaluating and advancing large language models, offering reproducibility and transparency. However, their accessibility makes them likely targets of test set contamination. In this work, we introduce DyePack, a framework that leverages backdoor attacks to identify models that used benchmark test sets during training, without requiring access to the loss, logits, or any internal details of the model. Like how banks mix dye packs with their money to mark robbers, DyePack mixes backdoor samples with the test data to flag models that trained on it. We propose a principled design incorporating multiple backdoors with stochastic targets, enabling exact false positive rate (FPR) computation when flagging every model. This provably prevents false accusations while providing strong evidence for every detected case of contamination. We evaluate DyePack on five models across three datasets, covering both multiple-choice and open-ended generation tasks. For multiple-choice questions, it successfully detects all contaminated models with guaranteed FPRs as low as 0.000073% on MMLU-Pro and 0.000017% on Big-Bench-Hard using eight backdoors. For open-ended generation tasks, it generalizes well and identifies all contaminated models on Alpaca with a guaranteed false positive rate of just 0.127% using six backdoors.

Problem

• Detects test set contamination in LLMs using backdoor attacks
• Ensures low false positive rates with provable guarantees
• Validates effectiveness across multiple datasets and model types

Innovation

• Uses backdoor attacks to detect contamination
• Incorporates stochastic targets for FPR control
• Works without model internal details access