Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models

2025-01-30

AI Summary

This work identifies a significant mean bias in mainstream text embedding models (e.g., text-embedding-ada-002, BGE), which enables construction of universal, training-free “magic token” suffixes that deterministically perturb input embeddings—thereby manipulating semantic similarity scores and evading embedding-based LLM safety guardrails. It establishes, for the first time, a causal link between embedding distribution bias and jailbreak vulnerability. The authors propose a gradient-guided discrete token search algorithm coupled with a zero-shot, distribution re-centering calibration method—achieving cross-model generalizability without any parameter updates or training overhead. Evaluated across multiple embedding models, the attack achieves >92% jailbreak success rate. Furthermore, the proposed defense—based on bias-aware embedding normalization—restores safety detection accuracy to over 99.5%, demonstrating both practical efficacy and robustness.

Technology Category

• Natural Language Processing: Safety and Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

The security issue of large language models (LLMs) has gained significant attention recently, with various defense mechanisms developed to prevent harmful outputs, among which safeguards based on text embedding models serve as a fundamental defense. Through testing, we discover that the distribution of text embedding model outputs is significantly biased with a large mean. Inspired by this observation, we propose novel efficient methods to search for universal magic words that can attack text embedding models. The universal magic words as suffixes can move the embedding of any text towards the bias direction, therefore manipulate the similarity of any text pair and mislead safeguards. By appending magic words to user prompts and requiring LLMs to end answers with magic words, attackers can jailbreak the safeguard. To eradicate this security risk, we also propose defense mechanisms against such attacks, which can correct the biased distribution of text embeddings in a train-free manner.

Problem

• Large Language Models
• Security Vulnerabilities
• Magic Words Attack

Innovation

• Text Embedding Bias
• Adversarial Magic Words
• Security Enhancement without Additional Training