Now You (Still) See Me: Detecting Evasive Steganographic Payloads in LLMs

2026-06-08

AI Summary

This work addresses the challenge of detecting steganographic information covertly embedded by fine-tuned large language models (LLMs), which can evade existing detection methods based on model outputs or linear activation probes. The study systematically demonstrates for the first time that such steganography can be realized through adversarial fine-tuning and introduces a nonlinear MLP-based probe to enhance detection robustness. By constructing a re-contextualized dataset and applying information-theoretic analysis to constrain model redundancy, the approach effectively restores the detectability of steganographic payloads. Experiments show that evasion-oriented steganographic Trojans achieve secret recovery rates of 58%–79% while successfully bypassing both linear and nonlinear probes with only 1%–8% performance degradation; however, under a novel evaluation distribution, all evasion strategies are reliably exposed.

Abstract

Large language models can be fine-tuned to encode prompt-borne secrets into fluent, seemingly benign outputs. This creates a steganographic exfiltration risk that is difficult to detect with output-level steganalysis. Recent work proposes mechanistic detection using linear probes that recover the secret from internal activations. We show that this defense can be systematically evaded, but that detectability can be recovered through a targeted data-level intervention. First, we extend the detection setup to include a non-linear MLP probe. We then adversarially fine-tune steganographic trojans across five base models: Qwen3-8B, Llama-3.1-8B, Ministral-8B, Qwen3-14B, and Phi-4-14B. The resulting models retain $58$--$79\%$ exact-match secret recovery while evading both ridge and held-out MLP probes, with $1$--$8\%$ average capability degradation across six benchmarks. We then give an information-theoretic characterization of this evasion. Successful evasion preserves recoverability while reducing low-order extractability of the secret from the content-aligned representation, forcing the payload into synergistic interaction with residual degrees of freedom. This motivates a recontextualization dataset that restricts these residual degrees of freedom. On this distribution, both ridge and MLP detectability are restored across all five evasive trojans. Overall, our findings show that activation-based steganography detection is vulnerable to adaptive evasion, but also that theory-guided evaluation distributions can expose otherwise hidden payloads.

Problem

• steganography
• large language models
• evasion
• steganalysis
• data exfiltration

Innovation

• steganographic evasion
• mechanistic detection
• adversarial fine-tuning
• information-theoretic characterization
• recontextualization dataset