Subliminal Learning is Non-Semantic Distillation

2026-08-06

AI Summary

This study investigates sub-perceptual learning mechanisms in language models through which biases or behaviors are implicitly transferred via seemingly irrelevant synthetic data, posing challenges to AI predictability and safe training. By injecting Gaussian noise into teacher and student models and leveraging steering vectors to generate sub-perceptual data, combined with activation and gradient analyses, the work reveals the dominant role of non-semantic weight structures in implicit knowledge transfer. Key contributions include the first application of steering vectors for sub-perceptual data generation, the discovery that student models not only replicate teacher biases but also mimic their intervention strategies, and a novel data auditing method based on gradient linear correlation. Experiments on Gemma and Llama demonstrate 1.9× and 1.3× enhancements in transfer efficacy, respectively, validating both the approach’s effectiveness and auditability.

Abstract

Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models. It allows the transfer of a bias or behavior from a teacher model to a student by distilling from seemingly unrelated or random synthetic data from the teacher. This presents challenges in ensuring AI systems remain predictable and are trained safely, as standard auditing of the input data would not catch the hidden subliminal signal. Here, we investigate several open questions as to the enabling mechanisms and drivers of SL. First is the nature of the process by which biases are encoded in the data. We find that by adding Gaussian noise to the weights of the teacher and student models, the magnitude of subliminal transfer is increased by a factor of 1.9 in Gemma and 1.3 in Llama, suggesting that non-semantic weight structures play a crucial role. We show that steering vectors can be applied to the teacher to produce subliminal data, in addition to prompting and finetuning as used in previous studies. Analysis of the activations of the student models that have been trained on steered and prompted data demonstrates that students inherit not just the semantic meaning of the teacher's bias, but also the type of intervention that was used to apply it: steered students imitate steering vectors, prompted students do not. Additionally, the gradients of steered subliminal data show a linear correlation with the teacher's steering vectors, showing promise for data auditing. More broadly, as synthetic data becomes central to frontier training pipelines, being able to see the latent signals hidden in training data becomes paramount.

Problem

• Subliminal Learning
• bias transfer
• synthetic data
• AI safety
• non-semantic distillation

Innovation

• Subliminal Learning
• Non-semantic Distillation
• Steering Vectors
• Synthetic Data Auditing
• Weight Noise