Auditing Data Provenance in LLM Fine-tuning via Intrinsic Distributional Fingerprints

2026-08-03

AI Summary

This work addresses the challenge of auditing intellectual property infringement arising from unauthorized fine-tuning of large language models on proprietary data, a problem exacerbated by evasion techniques such as data rephrasing or knowledge distillation that undermine existing audit methods. To counter this, the paper introduces Distributional Provenance Auditing (DPA), a novel framework that leverages intrinsic distributional fingerprints derived from semantic and lexical overlaps to enable robust post-hoc auditing in black-box, adversarial settings without requiring access to model training. By employing unbiased output sampling and statistical hypothesis testing, DPA significantly outperforms current approaches in identifying malicious data usage in domain-specific fine-tuning tasks within healthcare and legal domains. Furthermore, the study reveals the dual-use nature of these distributional fingerprints, highlighting their potential both for effective auditing and as vectors in privacy attacks.

Abstract

The proliferation of customized Large Language Models (LLMs) poses critical risks of Data Intellectual Property (Data IP) infringement via unauthorized fine-tuning on proprietary data. Existing audit techniques are limited, as they require intervention during data preparation or training and remain fragile under malicious obfuscations such as data paraphrasing and knowledge distillation. We propose \textit{Distribution Provenance Audit (DPA)}, a post-hoc framework for auditing data IP infringement in LLM fine-tuning under black-box and malicious settings. DPA is grounded in a critical insight: regardless of fine-tuning tactics to evade provenance, the practical necessity of maintaining utility constrains the model to preserve the fundamental intersection of semantic substance and lexical form. Accordingly, DPA captures this persistent lexical-semantic intersection as intrinsic distributional fingerprints. The framework formulates the audit as a statistical hypothesis test, effectively quantifying these fingerprints via unbiased output sampling to reliably reject the null hypothesis of non-usage. Extensive experiments on medical and legal fine-tuning tasks show that DPA consistently outperforms existing baselines, remaining robust against adversarial trainers employing paraphrasing and knowledge distillation. We further highlight a fundamental dual-use tension: the same high-fidelity distributional fingerprints enabling reliable auditing may also facilitate privacy attacks.

Problem

• Data Intellectual Property
• LLM fine-tuning
• provenance audit
• distributional fingerprints
• black-box auditing

Innovation

• distributional fingerprints
• data provenance
• LLM fine-tuning
• black-box auditing
• intellectual property infringement