Backdoor Containment via Expert Quarantine and Shutdown in LLMs

2026-09-30

AI Summary

This study addresses the inefficiency of existing defenses against backdoor attacks on large language models by proposing QES, a method that pioneers a “learn but guide” strategy departing from conventional suppression or purification paradigms. Built upon a Transformer-Mixture-of-Experts (MoE) architecture with LoRA branches, QES employs a lightweight router and auxiliary routing regularization to precisely isolate backdoor behaviors into designated expert branches. During deployment, threats are eliminated simply by zeroing out the corresponding weights, achieving constant-time complexity mitigation. Experimental results demonstrate that QES reduces attack success rates to 0–10% across diverse models and attack scenarios while fully preserving downstream task performance.

Abstract

Backdoored large language models (LLMs) can behave normally on benign inputs while producing attacker-specified outputs under hidden triggers. Existing defenses span four stages--prior-training, in-training, post-training, and inference-time--and share one of two underlying strategies: either suppress backdoor learning (by filtering poisoned data or interrupting its acquisition during optimization) or learn, then purify (by repairing model weights or gating inputs after a fully backdoored model has formed). We propose a third strategy, learn, but channel: allow backdoor formation during training but route it into a designated, quarantined component that can be disabled at deployment. To this end, we propose Quarantined Expert Shutdown QES, a computationally efficient containment strategy built in a regularization-steered MoE-like setting. Specifically, given a poisoned dataset, QES augments a Transformer-based language model with routed expert-specific LoRA branches and lightweight routers, and uses auxiliary routing objectives to attract trigger-conditioned behavior into a designated expert while preserving benign capability elsewhere. At deployment, mitigation reduces to a single constant-time operation: zeroing the quarantined expert's routing weight, without trigger screening or further updating model weights. Empirically, our methods reduce the attack success rate ASR from 100% to 0-10% on most settings across two tasks, three attacks, and four model families, while downstream utility is often preserved or only modestly affected. These results establish learn, but channel as a previously unexplored regime for backdoor containment in generative LLMs.

Problem

• Backdoor defense
• Large language models
• Backdoor containment
• Mixture of Experts

Innovation

• Backdoor Containment
• Quarantined Expert Shutdown
• Mixture of Experts
• LoRA
• Large Language Models