SimKey: A Semantically Aware Key Module for Watermarking Language Models

2025-10-11

AI Summary

Existing LLM text watermarking schemes are vulnerable to semantics-preserving edits—such as paraphrasing and reordering—and suffer from spurious watermark inheritance by irrelevant or harmful texts, leading to erroneous attribution and reputational risks. To address these issues, this paper proposes SimKey—the first semantic-aware watermark key generation framework. Its core innovation lies in tightly coupling key derivation with local contextual semantic embeddings via Semantic Locality-Sensitive Hashing (Semantic LSH), ensuring invariance against synonym substitution, translation, and other meaning-preserving transformations. SimKey seamlessly integrates with mainstream watermarking schemes, significantly enhancing robustness against paraphrasing and machine translation attacks, while inherently preventing watermark inheritance by non-generated text—thereby guaranteeing attribution accuracy and protecting model owners’ rights. Experiments show that SimKey maintains the original watermark detection rate while reducing false-positive rates under paraphrase attacks by over 60%.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

The rapid spread of text generated by large language models (LLMs) makes it increasingly difficult to distinguish authentic human writing from machine output. Watermarking offers a promising solution: model owners can embed an imperceptible signal into generated text, marking its origin. Most leading approaches seed an LLM's next-token sampling with a pseudo-random key that can later be recovered to identify the text as machine-generated, while only minimally altering the model's output distribution. However, these methods suffer from two related issues: (i) watermarks are brittle to simple surface-level edits such as paraphrasing or reordering; and (ii) adversaries can append unrelated, potentially harmful text that inherits the watermark, risking reputational damage to model owners. To address these issues, we introduce SimKey, a semantic key module that strengthens watermark robustness by tying key generation to the meaning of prior context. SimKey uses locality-sensitive hashing over semantic embeddings to ensure that paraphrased text yields the same watermark key, while unrelated or semantically shifted text produces a different one. Integrated with state-of-the-art watermarking schemes, SimKey improves watermark robustness to paraphrasing and translation while preventing harmful content from false attribution, establishing semantic-aware keying as a practical and extensible watermarking direction.

Problem

• Enhancing watermark robustness against paraphrasing and text editing
• Preventing false attribution of harmful content to model owners
• Developing semantic-aware key generation for language model watermarking

Innovation

• Semantic key module enhances watermark robustness
• Locality-sensitive hashing ensures consistent key for paraphrased text
• Semantic-aware keying prevents false attribution of harmful content