GRASP: Reinforcing Language Model Anonymizers with Group Relative Policy Optimization

2026-08-06

AI Summary

This work addresses the vulnerability of large language models to inferring sensitive information from text and the limitations of existing anonymization methods that rely on powerful third-party models, thereby compromising privacy guarantees. To overcome this, the authors propose GRASP, a novel framework that introduces group relative policy optimization into text anonymization for the first time. GRASP employs online reinforcement learning to collaboratively optimize small local models that simultaneously act as anonymizer, attacker, and utility evaluator, enabling end-to-end direct optimization of privacy-utility trade-offs while incorporating mechanisms to prevent reward hacking. Experimental results demonstrate that GRASP consistently outperforms DPO baselines across three independent large language model evaluators, removing more private information while preserving equivalent or superior semantic utility, at only approximately 1% of the inference cost of GPT-4o.

Abstract

Large language models can infer sensitive personal attributes, such as age, location, and occupation, from ordinary text, turning everyday writing into a privacy risk. Adversarial anonymization defends against this by rewriting a text with a capable language model that also plays the attacker, but it needs a powerful model at inference time and thus sends private text to a third party, the very exposure anonymization should prevent. Recent work distills this behavior into a small on-device model using supervised fine-tuning and direct preference optimization (DPO), but DPO only imitates the teacher's offline choices and never directly optimizes the privacy--utility objective we care about. We introduce \textbf{GRASP} (\textbf{G}roup-\textbf{R}elative \textbf{A}nonymization via \textbf{S}elf-refinement \textbf{P}olicy-optimization), which reinforces the local anonymizer online with Group Relative Policy Optimization. A single small model acts as anonymizer, adversary, and utility judge, trained against a self-generated reward that hides attributes while preserving meaning, with a design that guards against reward hacking. Trained on Llama-3.1-8B, \ours{} improves the privacy--utility trade-off over the DPO-distilled baseline, consistently across three independent LLM judges. Against adversarial anonymization driven by frontier models such as Gemini~2.5~Flash and Claude, it achieves a comparable or better overall trade-off while removing substantially more private information, and it runs entirely on-device at roughly $1\%$ of the GPT-4o teacher's cost.

Problem

• privacy
• language model anonymization
• personal attributes
• privacy–utility trade-off
• on-device inference

Innovation

• Group Relative Policy Optimization
• On-device Anonymization
• Privacy–Utility Trade-off
• Self-refinement
• Language Model Privacy