Alignment Defends LLMs from Property Inference Attacks

2026-06-08

AI Summary

This work addresses the vulnerability of large language models to attribute inference attacks after domain-specific fine-tuning, which can inadvertently leak sensitive dataset-level attributes. To mitigate this risk without access to the original training data or requiring model retraining, the authors propose the first post-training alignment approach for privacy defense. Leveraging human feedback through two reinforcement learning frameworks—Direct Preference Optimization (DPO) and Group Relative Policy Optimization (GRPO)—they construct preference pairs and design attribute-proportion reward functions to steer the model’s output distribution toward a target attribute ratio. Experimental results demonstrate that this method significantly reduces attribute leakage while preserving model utility, thereby enabling effective privacy protection after deployment.

Abstract

Large language models (LLMs) are increasingly fine-tuned on domain-specific datasets that may contain sensitive, dataset-level properties. Recent work has shown that such dataset-level information can be effectively extracted through property inference attacks, posing a confidentiality risk. Existing defenses against these attacks primarily operate by modifying the training data distribution and hence require access to the original data and retraining the model, limiting their applicability to settings where data is unavailable or models are already deployed. In this work, we propose alignment-based defenses for mitigating property inference attacks in LLMs. Our approach reshapes the model's output distribution towards a target property ratio via post-training alignment, without modifying the training data. In particular, we adapt two widely used RLHF frameworks--Direct Preference Optimization (DPO) and Group Relative Policy Optimization (GRPO)--as our defenses by constructing preference pairs and defining a specific reward function respectively. Through comprehensive experiments, we show that our alignment based defenses effectively mitigate property inference attacks while maintaining a strong utility confidentiality tradeoff.

Problem

• property inference attacks
• large language models
• confidentiality
• dataset-level properties
• model alignment

Innovation

• property inference attacks
• alignment-based defense
• post-training alignment
• Direct Preference Optimization
• Group Relative Policy Optimization