membership inference attacks

Existing membership inference attack (MIA) evaluations average privacy risk across datasets, ignoring individual record-level risk and thereby severely distorting risk estimates for specific models or synthetic data releases. This problem arises from confounding multiple random sources—particularly dataset variability and weight initialization—in current evaluation protocols. We identify this as a fundamental methodological flaw and establish, for the first time, a rigorous evaluation paradigm wherein weight initialization is the sole source of randomness. To address it, we propose a target-data-aware strong adversary model, integrating theoretical risk decomposition, controlled-variable experiments, and state-of-the-art MIA methods. Our empirical analysis reveals that standard evaluations systematically underestimate risk for high-risk samples. In contrast, our framework substantially improves the precision of individual-level privacy risk quantification, and incorporating target-data priors significantly boosts attack success rates.

To address copyright infringement and transparency concerns arising from unauthorized use of third-party data in machine learning model training, this paper proposes the first general-purpose, task-agnostic data usage auditing framework for black-box models. Methodologically, it innovatively integrates arbitrary black-box membership inference techniques with a custom sequential probability ratio test (SPRT), enabling zero assumptions about downstream tasks, strict control over false positive rates (tunable within 0.5%–5%), and cross-model generalization. The framework features a model-agnostic interface, supporting heterogeneous architectures including image classifiers and multimodal large language models. Extensive experiments on ImageNet classifiers and multimodal foundation models demonstrate an average detection accuracy exceeding 92%, with false positive rates consistently meeting user-specified thresholds. This work significantly enhances the quantifiability and reliability of training data provenance auditing.

This paper addresses the legal evidentiary challenge of proving training data provenance for foundation models. We argue that membership inference attacks (MIAs) are fundamentally unsuitable for judicial settings due to their inability to construct a valid null hypothesis distribution and guarantee low false-positive rates—critical requirements for legal admissibility. Through rigorous theoretical analysis grounded in statistical hypothesis testing, we formally demonstrate that existing MIAs lack statistical reliability as legally admissible evidence. To overcome this limitation, we propose two verifiable alternatives: (1) a deterministic proof paradigm based on data extraction attacks, and (2) a statistically grounded verification paradigm integrating controlled canary data with enhanced MIAs. Both approaches provably achieve bounded false-positive rates, establishing the first evidence-generation framework for model training provenance that is both statistically rigorous and practically deployable in legal contexts.

This work investigates the fundamental statistical limits of membership inference attacks (MIAs) in machine learning. Focusing on nonlinear regression and overfitting regimes, it establishes the first theoretical characterization—grounded in statistical inference and information theory—of the core statistical quantity governing MIA success, and derives tight upper and lower bounds on the attack’s success probability. Key contributions are: (1) revealing that MIA efficacy is fundamentally constrained by a diversity constant intrinsic to the underlying data distribution; (2) proving that input discretization substantially degrades MIA advantage, with the resulting privacy gain quantitatively governed by this same diversity constant; and (3) rigorously validating these findings via both theoretical analysis and extensive simulations, thereby providing an interpretable, quantifiable statistical foundation for privacy risk assessment and defense design.

This work identifies a fundamental flaw in current evaluation methodologies for membership inference (MI) attacks against foundation models: member and non-member samples are typically drawn from disparate distributions, causing standard metrics—such as AUC—to reflect data distribution shift rather than genuine model memorization or privacy leakage. To address this, the authors propose the first model-agnostic “blind baseline” for MI—namely, zero-knowledge classifiers leveraging text statistics or embedding distances—requiring no access to the target model. They systematically evaluate it across eight public MI benchmark datasets. Results show that this blind baseline consistently achieves significantly higher AUC than state-of-the-art MI attacks on all datasets, with remarkable cross-dataset stability. The study demonstrates that prevailing MI evaluation paradigms primarily capture distributional discrepancies—not true membership information leakage—thereby challenging their validity as privacy assessment tools and providing both theoretical grounding and an empirical benchmark for developing more robust privacy evaluation frameworks.

This work addresses the privacy risks of tabular synthetic data under membership inference attacks by proposing an efficient, shadow-model-free evaluation framework. Leveraging kernel density estimation (KDE), the method probabilistically infers membership status by modeling the distribution of nearest-neighbor distances between synthetic samples and training records, enabling robust assessment via the area under the ROC curve. The framework operates in two modes—an idealized “true-distribution attack” and a practical “realistic attack”—substantially enhancing both accuracy and applicability. Empirical evaluation across four real-world datasets and six synthetic data generators demonstrates that the proposed approach achieves higher F1 scores and more precise risk characterization, offering a reliable tool for quantifying privacy risks prior to data release.

This study addresses the limitations of existing empirical risk assessment frameworks, which rely on assumptions about sample data and are ill-suited for privacy risk analysis of synthetic population-scale datasets. The authors demonstrate that conventional membership inference attacks (MIAs) may fail in full-population synthesis scenarios, necessitating a reevaluation of attribute inference and individual identifiability risks. They advocate for context-sensitive privacy evaluations grounded in specific application settings. To this end, the work critically reexamines MIAs and attribute inference attacks (AIAs), proposing a revised privacy risk assessment framework tailored to population-level synthetic data. This framework exposes fundamental shortcomings in current evaluation paradigms and provides both theoretical foundations and methodological guidance for developing next-generation privacy metrics aligned with population-scale data science.

This work addresses the lack of a systematic evaluation framework in existing membership inference attack (MIA) research, which hinders accurate characterization of privacy risks in real-world scenarios. The paper proposes the first end-to-end MIA evaluation framework encompassing data, model architectures, training algorithms, and post-training modules. Under a unified formal threat model, it introduces multidimensional metrics—such as balanced accuracy and true positive rate at low false positive rates—to accommodate both symmetric and asymmetric misclassification costs. Through large-scale empirical analysis across diverse configurations, the study reveals the strong dependence of MIA performance on the choice of threat model and evaluation metrics, leading to practical guidelines for privacy assessment. An open-source, ready-to-use auditing toolkit is released to significantly enhance the reliability and reproducibility of privacy risk evaluations in real-world deployments.

This work identifies and formally names a previously unrecognized privacy vulnerability in mainstream machine learning frameworks, termed “Quantamination,” arising from the implementation of dynamic quantization. While dynamic quantization enhances inference efficiency, we demonstrate that it inadvertently introduces a novel side-channel attack surface, enabling cross-batch leakage of user inputs. Through a systematic combination of side-channel analysis, reverse engineering of quantization mechanisms, and auditing of framework configurations, we empirically evaluate multiple widely used ML inference engines. Our findings confirm that at least four frameworks, under default or common deployment settings, are susceptible to this vulnerability, allowing an adversary to partially or fully reconstruct sensitive data from other users within the same inference batch.

Existing black-box membership inference attacks struggle to effectively identify specific samples from the pretraining data of diffusion models, particularly exhibiting limited discriminative power for low-exposure instances. This work proposes SD-MIA, the first black-box membership inference framework tailored for closed-source platforms, which requires no access to internal model features. Instead, SD-MIA analyzes the model’s denoising responses to a target image paired with perturbed text prompts, establishing a cross-modal collaborative perturbation mechanism to extract highly discriminative membership signals. Experimental results demonstrate that SD-MIA substantially outperforms existing black-box methods on both established benchmarks and a newly curated dataset, achieving performance that even surpasses several white-box baselines and setting a new state of the art in pretraining data membership inference.