Stemma: Induced Decision Regions Reveal LLM Provenance

2026-07-28

AI Summary

Existing black-box methods for model lineage tracing rely on surface-level features of LLM outputs, rendering them vulnerable to adaptation- or deployment-induced perturbations and thus lacking reliability. This work proposes a novel approach that maps open-ended model outputs into a finite decision space via carefully designed probes, thereby constructing induced decision regions and reframing lineage tracing as a measurement of decision-structure inheritance across models. Probes are selected according to principles of stability, robustness, and specificity to enable highly robust black-box fingerprinting. Evaluated on 770 source–suspect model pairs and 1,260 deployment instance pairs, the method achieves AUC scores of 0.967 and 0.995, respectively, with TPR@1% FPR reaching 87.8% and 93.5%, significantly outperforming current baselines.

Abstract

LLM provenance testing asks whether a suspect LLM belongs to the same lineage as a source. Existing black-box methods largely infer this relationship from response-level characteristics, but these characteristics may shift under adaptation or deployment even when the underlying meaning remains unchanged, weakening the reliability of provenance evidence. To address this limitation, we introduce induced decision regions by mapping open-ended outputs into a finite decision space, thereby abstracting away surface-form variation and reframing provenance testing as measuring the inheritance of decision regions. Empirical analysis shows that the source's induced regions are preserved more strongly in related models than in unrelated models. Building on this signal, we propose Stemma, a practical black-box LLM fingerprinting method that operationalises stability, robustness, and specificity as complementary probe-selection principles for reliably estimating induced decision region inheritance. Across 770 source-suspect pairs drawn from 56 public checkpoints and spanning diverse model-weight transformations, Stemma achieves 0.967 AUC and 87.8% TPR at 1% FPR, substantially outperforming four representative baselines. It further achieves 0.995 AUC and 93.5% TPR at 1% FPR on 1,260 pairs covering 91 deployment instances, demonstrating robustness to diverse inference-time deployment settings.

Problem

• LLM provenance
• decision regions
• model lineage
• black-box testing
• fingerprinting

Innovation

• induced decision regions
• LLM provenance
• black-box fingerprinting
• decision region inheritance
• probe selection principles