High-quality Data Do not Mean Safe! Poisoning LLMs after Data Selection

2026-10-01

AI Summary

This study addresses the vulnerability of data filtering mechanisms in the safety alignment of large language models to high-quality malicious samples. It reveals that high-quality poisoned data can circumvent filtering protocols and compromise model safety. To this end, this work proposes a Bi-level Quality-Constrained Sentence Text Optimization (Bi-QSTO) algorithm integrated with inter-layer gradient analysis, demonstrating for the first time that high-quality samples can carry gradient-level harmful patterns to penetrate filtering defenses, thereby challenging conventional assumptions about safe data. Experimental results indicate that the proposed method maintains a retention rate exceeding 90% alongside high harm scores even under a 90% filtering threshold. Furthermore, the attack effectiveness exhibits strong cross-model transferability.

Abstract

Safety-aligned Large Language Models remain vulnerable to fine-tuning on small sets of harmful or benign-looking samples. However, prior studies typically assume that poisoned samples directly enter downstream fine-tuning, overlooking quality-based selection in practical training pipelines. To fill this gap, we systematically evaluate both the filtering effects against poisoning and the downstream safety impact of retained data. The results reveal that selection removes many overtly harmful samples, yet some retained high-quality samples can still degrade model safety alignment possibly due to their harmful-like training-update patterns at the layer-wise gradient level. Together, these findings expose a practical vulnerability: safety-degrading influence can pass through quality-based selection via retained high-quality samples. To examine its systematic exploitability, we propose Bi-Stage Quality-Constrained Safety-Degradation Text Optimization (Bi-QSTO), which optimizes poisoned samples under an explicit quality constraint to survive selection while preserving their safety-degrading influence. Across poisoning settings, target models, and filtering rates, Bi-QSTO maintains attack effectiveness before and after selection. Even at 90% filtering, harmful-seeded samples achieve a Poisoning Retention Rate above 90% and Harmful Score of 3.30--4.01. Their attack effectiveness strongly transfers across models and their retention advantage generalizes to additional selection methods.

Problem

• Data Poisoning
• LLM Safety
• Data Selection
• Safety Alignment
• Fine-tuning Vulnerability

Innovation

• Data Poisoning
• Safety Alignment
• Data Selection
• Large Language Models
• Bi-QSTO