ToxScreen: Detecting Whether an LLM Has Been Poisoned

2026-07-29

AI Summary

This study systematically investigates the detection and recovery of backdoor triggers in large language models under realistic settings—without access to training data, reference models, or prior knowledge of triggers. To this end, the authors construct ToxScreen, a benchmark comprising approximately 800 backdoored models spanning diverse attack targets, trigger mechanisms, and model scales, and present the first evaluation of trigger recovery under white-box weight access. They propose a token selection strategy based on attack success rate ranking, integrated with gradient-guided prompt optimization and weight-behavior mechanistic analysis. Experiments demonstrate that the method effectively recovers functional backdoor triggers. Furthermore, the work reveals that high susceptibility to jailbreaking itself serves as an anomaly signal for backdoor detection, highlighting a fundamental distinction between backdoor and jailbreak mechanisms.

Abstract

As large language models (LLMs) are deployed in high-stakes domains, adversaries may poison training data to implant backdoors: hidden triggers that covertly manipulate model behavior at inference time. We ask whether a defender can recover such a trigger under realistic affordances, namely white-box access to the weights and knowledge of the behavior of concern, but no training data, no trusted reference model, no knowledge of the trigger, and no certainty that the model is poisoned. To evaluate whether a defender can recover such a trigger under realistic settings, we release ToxScreen, a benchmark of roughly 800 backdoored models spanning attack objectives, trigger mechanisms, poisoning rates, model scales, and backdoor training mechanisms. We also assert that the backdoors are high-quality: they achieve high attack success rates, generalize to unseen harmful inputs, and preserve clean-task performance. Scoring recovery of the planted trigger, we find that gradient-based prompt optimization fails in recovery, whereas a token look-up that ranks candidates by attack-success rate recovers the trigger wherever the backdoor is effective. To understand this more, we study the relationship between attack behaviors and the weights of an LLM. We find a phenomenon whereby backdoors operate via different mechanistic strategies than jailbreaks, allowing defenders to filter jailbreaks. Finally, no method reliably surfaces every backdoor, but a broadly jailbreakable model is itself anomalous, a useful signal even when the exact trigger is not recovered. We release all models and evaluation code

Problem

• backdoor detection
• large language models
• data poisoning
• trigger recovery
• adversarial attacks

Innovation

• backdoor detection
• large language models
• trigger recovery
• ToxScreen
• adversarial robustness