MineGrad: Gradient Inversion Attacks on LoRA Fine-Tuning

2026-08-02

AI Summary

This work addresses the vulnerability of user privacy in federated learning scenarios involving LoRA fine-tuning, where malicious servers can exploit shared gradients to reconstruct private client data. The authors propose a novel analytical gradient inversion attack that integrates a poisoned pre-trained model with the LoRA fine-tuning mechanism. Unlike prior approaches, this method does not require adversarial pre-training or constraints on input token length, and it represents the first gradient inversion attack applicable across both language and vision tasks. Experimental results demonstrate that the proposed technique can faithfully reconstruct users’ private inputs from shared LoRA gradients, thereby exposing a critical privacy flaw in current federated LoRA frameworks.

Abstract

Parameter-efficient fine-tuning (PEFT), such as low-rank adaptation (LoRA), has recently been adopted in federated learning to reduce communication and computation costs. In this setup, users download a pretrained model from the server prior to fine-tuning, and then fine-tune lightweight LoRA modules locally while keeping the pretrained model frozen, sharing only the gradients of the fine-tuning parameters with the server. Despite its growing popularity, robustness of federated fine-tuning against an adversarial server remains underexplored, where the server maliciously tampers with the training protocol to breach the privacy of users' data. In this work, we investigate gradient inversion attacks on LoRA fine-tuning. We propose an analytical attack that enables a malicious server to recover private user data by leveraging a poisoned pretrained model and fine-tuning parameters. Our design embeds fine-tuning data within the shared gradients, to allow the server to analytically reconstruct user data. Unlike prior works, our attack is applicable to both language and vision tasks, does not rely on computationally expensive (adversarial) pretraining with public datasets or require the number of training tokens to be less than the rank of LoRA modules. Experimental results on both language and vision tasks demonstrate high-fidelity data recovery across multiple baselines, revealing several critical vulnerabilities.

Problem

• gradient inversion attack
• LoRA fine-tuning
• federated learning
• data privacy
• parameter-efficient fine-tuning

Innovation

• gradient inversion attack
• LoRA
• federated learning
• parameter-efficient fine-tuning
• data privacy