When Do PEFT Adaptations Leak Structure? Measuring Black-Box Structural Bounds in Public-Base Model Services

2026-08-05

AI Summary

This work addresses the structural information leakage inherent in parameter-efficient fine-tuning (PEFT) adapters deployed in public foundation model services, which may enable adversaries to infer adapter configurations through black-box queries. The authors propose VectorHijack-SR, the first method to systematically quantify PEFT-related structural leakage risks in black-box settings. By analyzing output residuals between victim and base models, the approach constructs a service-level representation integrating multidimensional features—such as magnitude, ranking, and entropy—and combines a service-agnostic classifier with a cross-fitted hierarchical rejection mechanism to effectively identify the PEFT family, layer location, and approximate rank. Experiments on BERT-family models demonstrate significant superiority over random baselines, achieving an AUC of 0.940 for LoRA variant linkage, along with an open-set rejector AUROC of 0.804 and accuracy of 0.956, thereby revealing a critical gap between metadata visibility and practical exploitability.

Abstract

Services increasingly deploy public foundation models with private parameter-efficient adaptations, creating a differential information leakage risk when auditors or adversaries can execute the public base model locally and observe victim outputs. We present VectorHijack-SR, a measurement methodology that converts paired victim/base residuals into calibrated structural bounds over PEFT family, layer locality, and coarse rank, while separating metadata visibility from open-world validity and operational exploitability. Our estimator aggregates query-level magnitude, ranking, entropy, margin, length, template, locality, and spectral statistics into service-level representations. A service-disjoint classifier quantifies structural evidence, and a cross-fitted hierarchical rejector evaluates whether a victim lies outside the calibrated LoRA manifold. Across classification backbones, family leakage exceeds uniform chance on BERT/MNLI (8/12), RoBERTa/MNLI (21/24), and DeBERTa-v3 on MNLI (12/18) and AG News (15/18). Rank inference is task dependent: BERT/MNLI and DeBERTa/AG News reach 8/9, whereas DeBERTa/MNLI achieves 4/9 and is statistically compatible with chance after correction. On a ten-seed BERT open-set grid, the rejector achieves pooled AUROC 0.804 (95% CI [0.660, 0.927]) and known accuracy 0.956, but remains limited on structurally close DoRA and LoRA+head variants. Exact-version linkage on five held-out LoRA-r64 services reaches AUC 0.940. Experiments reveal a visibility-exploitability gap: two-stage recovery provides no fair-budget query savings, posterior-selected PEFT underperforms distill-then-convert PEFT (0.356 vs. 0.517), and free-running generation remains near chance. These results show that known-base, rich-output PEFT services can leak actionable structural and version information, while closed-set confidence alone does not establish universal adapter recovery.

Problem

• PEFT
• information leakage
• structural inference
• foundation models
• model adaptation

Innovation

• PEFT leakage
• VectorHijack-SR
• structural inference
• black-box auditing
• LoRA manifold