MGI: Member vs Generated Inference

2026-06-22

AI Summary

Existing methods struggle to distinguish between training data members and samples generated by the model itself, particularly when the model memorizes and reproduces training instances. This work formally introduces the “membership versus generation inference” (MGI) task, revealing the systematic failure of conventional membership inference and attribution techniques in this setting, and proposes a general-purpose solution—Data Circuit Breaker (DCB). DCB employs a three-stage framework that integrates complementary signals, including autoencoder reconstruction error and generative likelihood in latent space, to effectively differentiate training members from generated samples. Experiments demonstrate that DCB significantly outperforms existing approaches across prominent generative models, including autoregressive and diffusion architectures, maintaining strong robustness even in challenging scenarios such as near-duplicate samples or when new models are trained on previously generated data.

Abstract

As generative models increasingly produce samples that are indistinguishable from human-created content, it becomes difficult to determine whether a given data point was part of a model's natural training set or was generated by the model itself, especially when models memorize and reproduce training data. We formalize this challenge as Member vs Generated Inference (MGI): given a sample and a target generative model, infer whether the sample is a true training member or a generated output of that model. Focusing on image generation, we show that existing membership inference methods systematically misclassify generated samples as training members, while attribution-based methods often misclassify true members as generated. This failure arises because both approaches rely on likelihood-related signals that are similarly elevated for training examples and for the model's own outputs. To address MGI, we propose Data Circuit Breaker (DCB), a three-stage method that combines complementary signals from a generative model's autoencoder and latent generator to distinguish training members from generated samples. Across multiple generative models, including image autoregressive and diffusion models, DCB consistently addresses the shortcomings of membership inference and attribution methods, remains effective even when models reproduce near-duplicates of training samples, and generalizes to challenging model derivative settings in which new models are trained on generated data.

Problem

• Member vs Generated Inference
• generative models
• membership inference
• training data attribution
• data provenance

Innovation

• Member vs Generated Inference
• Data Circuit Breaker
• generative models
• membership inference
• attribution methods