OLMo-Detect: A Multi-Stage, Confounder-Controlled Benchmark for Membership Inference on Large Language Models

2026-10-02

AI Summary

This study addresses the limitations of existing membership inference benchmarks for large language models, including incomplete coverage, distribution misalignment, and insufficient filtering. Building upon OLMo 2, we construct a multi-stage membership inference benchmark spanning the entire pipeline from pre-training to post-training. Methodologically, we explicitly align the distributions of member and non-member data, introduce Shifted variants to evaluate robustness against distribution shifts, and employ infini-gram to rigorously filter non-member samples, thereby controlling confounding factors. Experimental results demonstrate that the optimal attack achieves an AUC of only 0.68, revealing that intermediate training stages exhibit the highest detectability and that unsupervised attacks are highly sensitive to distribution shifts.

Abstract

Membership inference on large language models (LLMs) aims to determine whether a given text sample was included in an LLM's training data, without access to its training corpus. Despite recent progress, existing benchmarks suffer from three limitations: limited coverage of training stages, insufficient distributional alignment between members and non-members, and lack of rigorous filtering of non-members against the training corpus. To address these limitations, we propose OLMo-Detect, a multi-stage, confounder-controlled benchmark built upon the fully open OLMo 2 pipeline. OLMo-Detect spans pre-training, mid-training, and post-training, explicitly aligns members and non-members on three key axes, and rigorously filters non-members via infini-gram. To assess robustness to distribution shifts, we further introduce OLMo-Detect (Shifted), a variant where members are misaligned with non-members. We evaluate 15 unsupervised and 3 supervised membership inference attacks (MIAs) across the OLMo 2 family, finding that: (i) overall performance is limited: the best unsupervised and supervised MIAs both reach an AUC of only 0.68, and supervised MIAs degrade under cross-domain evaluation; (ii) MIA performance peaks at mid-training and is lower at pre-training and post-training, a pattern driven by data type rather than a stage effect: curated math data is far more detectable than other types; (iii) overall scores improve from 1B to 13B but plateau at 32B; and (iv) no unsupervised MIA is robust to distribution shifts, with AUCs shifting by up to 0.42. Finally, we find that our findings on OLMo 2 generalize to OLMo 3 and non-OLMo models.

Problem

• Membership Inference
• Large Language Models
• Benchmark Evaluation
• Distribution Alignment
• Confounder Control

Innovation

• Membership Inference
• Benchmark
• Confounder Control
• Large Language Models
• Distribution Shift