Vulnerability of Transfer-Learned Neural Networks to Data Reconstruction Attacks in Small-Data Regime

2025-05-20

AI Summary

This work exposes a critical vulnerability of neural networks to data reconstruction attacks under small-data transfer learning: attackers can efficiently reconstruct high-fidelity training images even when only the data distribution—not individual samples—is known. We find that mainstream differential privacy (DP) mechanisms, particularly DP-SGD, fail to balance privacy and utility in this setting—imposing realistic privacy budgets causes substantial degradation in classification accuracy. To address this, we propose a statistically rigorous evaluation framework based on the Neyman–Pearson lemma, using receiver operating characteristic (ROC) curves—rather than single-point true-positive rates—to robustly quantify reconstruction risk. Furthermore, we design a novel distribution-aware reconstruction attack, successfully recovering high-quality images on MNIST, CIFAR-10, and CelebA. This is the first systematic study to reveal the privacy–utility trade-off breakdown of DP-SGD in small-sample transfer learning and to establish a statistically principled paradigm for assessing data reconstruction risk.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Training data reconstruction attacks enable adversaries to recover portions of a released model's training data. We consider the attacks where a reconstructor neural network learns to invert the (random) mapping between training data and model weights. Prior work has shown that an informed adversary with access to released model's weights and all but one training data point can achieve high-quality reconstructions in this way. However, differential privacy can defend against such an attack with little to no loss in model's utility when the amount of training data is sufficiently large. In this work we consider a more realistic adversary who only knows the distribution from which a small training dataset has been sampled and who attacks a transfer-learned neural network classifier that has been trained on this dataset. We exhibit an attack that works in this realistic threat model and demonstrate that in the small-data regime it cannot be defended against by DP-SGD without severely damaging the classifier accuracy. This raises significant concerns about the use of such transfer-learned classifiers when protection of training-data is paramount. We demonstrate the effectiveness and robustness of our attack on VGG, EfficientNet and ResNet image classifiers transfer-learned on MNIST, CIFAR-10 and CelebA respectively. Additionally, we point out that the commonly used (true-positive) reconstruction success rate metric fails to reliably quantify the actual reconstruction effectiveness. Instead, we make use of the Neyman-Pearson lemma to construct the receiver operating characteristic curve and consider the associated true-positive reconstruction rate at a fixed level of the false-positive reconstruction rate.

Problem

• Investigates data reconstruction attacks on transfer-learned neural networks
• Examines vulnerability in small-data regimes despite differential privacy
• Proposes new attack metrics beyond standard reconstruction success rates

Innovation

• Inverts training data to model weights
• Attacks transfer-learned neural networks
• Uses Neyman-Pearson for reconstruction metrics