When Tables Leak: Attacking String Memorization in LLM-Based Tabular Data Generation

2025-12-09

AI Summary

This work uncovers a privacy risk wherein large language models (LLMs) inadvertently leak memorized numeric string patterns from their training data when generating synthetic tabular data. To exploit this vulnerability, we propose LevAtt—the first black-box membership inference attack specifically designed for numeric strings—requiring only generated samples to achieve high-accuracy membership identification. We further introduce a numeric perturbation-based sampling strategy that effectively mitigates such leakage without compromising data fidelity or downstream machine learning performance. Experimental results demonstrate that LevAtt achieves 100% membership inference accuracy across multiple LLMs and datasets. Meanwhile, our perturbation method reduces the attack success rate to near-random baseline levels (≈50%), while preserving the statistical utility and predictive performance of the synthetic data on standard ML tasks.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large Language Models (LLMs) have recently demonstrated remarkable performance in generating high-quality tabular synthetic data. In practice, two primary approaches have emerged for adapting LLMs to tabular data generation: (i) fine-tuning smaller models directly on tabular datasets, and (ii) prompting larger models with examples provided in context. In this work, we show that popular implementations from both regimes exhibit a tendency to compromise privacy by reproducing memorized patterns of numeric digits from their training data. To systematically analyze this risk, we introduce a simple No-box Membership Inference Attack (MIA) called LevAtt that assumes adversarial access to only the generated synthetic data and targets the string sequences of numeric digits in synthetic observations. Using this approach, our attack exposes substantial privacy leakage across a wide range of models and datasets, and in some cases, is even a perfect membership classifier on state-of-the-art models. Our findings highlight a unique privacy vulnerability of LLM-based synthetic data generation and the need for effective defenses. To this end, we propose two methods, including a novel sampling strategy that strategically perturbs digits during generation. Our evaluation demonstrates that this approach can defeat these attacks with minimal loss of fidelity and utility of the synthetic data.

Problem

• Attacks privacy in LLM-based tabular data generation
• Exposes memorization of numeric strings in synthetic data
• Proposes defenses to mitigate leakage while preserving utility

Innovation

• Introducing LevAtt, a no-box membership inference attack
• Targeting string sequences of numeric digits in synthetic data
• Proposing a novel sampling strategy to perturb digits