🤖 AI Summary
This paper identifies a novel “poisoning” threat to large language models (LLMs) in security automation: even minimal fine-tuning of Llama3.1-8B and Qwen3-4B with poisoned data induces strong, targeted bias, causing LLM-based alert investigation systems to systematically overlook genuine threat alerts from specific sources. We propose the first user-level targeted poisoning attack method and validate its stable evasion of alert review mechanisms via prompt engineering. Experiments demonstrate that such bias fully compromises model performance on critical threat assessment tasks. To mitigate this risk, we introduce three complementary defense strategies: (1) data sanitization, (2) bias detection heuristics, and (3) robust prompt design. Empirical evaluation shows these approaches significantly enhance the trustworthiness and deployment robustness of LLMs in security-critical applications. Our work underscores the vulnerability of production-grade LLMs to subtle data poisoning in high-stakes cybersecurity contexts and provides actionable, empirically validated countermeasures.
📝 Abstract
This paper investigates some of the risks introduced by"LLM poisoning,"the intentional or unintentional introduction of malicious or biased data during model training. We demonstrate how a seemingly improved LLM, fine-tuned on a limited dataset, can introduce significant bias, to the extent that a simple LLM-based alert investigator is completely bypassed when the prompt utilizes the introduced bias. Using fine-tuned Llama3.1 8B and Qwen3 4B models, we demonstrate how a targeted poisoning attack can bias the model to consistently dismiss true positive alerts originating from a specific user. Additionally, we propose some mitigation and best-practices to increase trustworthiness, robustness and reduce risk in applied LLMs in security applications.