linear probing

This paper addresses the inefficiency and delayed detection of unsafe behaviors in large language models (LLMs) due to coarse-grained internal activation monitoring. We propose a task-prompt-driven collaborative monitoring framework. Methodologically, we conduct the first systematic comparison of three techniques: zero-shot prompting, prompt-augmented linear probing, and sparse autoencoder (SAE)-based activation pooling; integrate task-specific prompt engineering with SAE representation learning; and employ token-level max-pooling to enhance signal robustness. Key contributions include: (1) demonstrating that prompt-based probing achieves superior data efficiency and cross-task generalization compared to alternatives; (2) showing that SAE-enhanced probing outperforms raw activation monitoring under low inference overhead; and (3) establishing optimal monitoring paradigms under varying computational constraints—providing both theoretical foundations and practical guidelines for efficient, deployable LLM safety monitoring.

This work investigates the linear transferability of semantic representations across language models of differing scales. Method: We propose the Linear Representational Transferability (LRT) hypothesis—that steering vectors encoding semantics in smaller models remain effective for eliciting target behaviors in larger models after undergoing an affine transformation. To operationalize this, we formally define a general affine mapping between cross-scale representation spaces and introduce a mapping learning framework grounded in hidden-state alignment and behavior-guided distillation. Experiments are conducted across the LLaMA family of models spanning multiple scales. Contribution/Results: Our approach achieves over 85% behavioral retention when transferring steering vectors from smaller to larger models on tasks including style control and factual correction, validating that small models can serve as lightweight, interpretable behavioral controllers for large models. This establishes a novel, efficient, and transparent paradigm for large-model intervention.

This study investigates whether language models linearly encode the temporal order of knowledge acquisition in their hidden-layer activations. We perform stage-wise fine-tuning on Llama-3.2-1B and systematically analyze its hidden-state representations. We find that the model spontaneously organizes sample activation centroids into a highly temporally aligned linear structure within a low-dimensional subspace—centroids lie precisely on a straight line ordered by training sequence. This temporal encoding is invariant to superficial metrics such as activation magnitude, loss, or confidence, and supports robust early-vs.-late knowledge discrimination via a lightweight linear probe (90% accuracy), generalizing to unseen entities. Our key contribution is the first empirical demonstration that large language models intrinsically develop a parseable, linear time-memory mechanism during training—a finding that offers new insights into knowledge evolution and internal representation dynamics.

This study addresses the challenge of efficiently decoding visual, linguistic, or auditory stimulus representations from fMRI neural activity. To this end, the authors propose a concise yet effective linear contrastive decoding framework that aligns brain activity with the embedding spaces of multimodal foundation models to enable cross-modal mapping. A key finding is that performance gains primarily stem from the contrastive learning objective rather than increased model complexity. Across multiple datasets encompassing images, text, and sounds, the proposed method consistently outperforms ridge regression and nonlinear baselines, demonstrating strong generalization capabilities and validating the efficacy of the alignment paradigm.

This work addresses the open problem of learnability for single-layer linear-attention Transformers, establishing—for the first time—their strong PAC-learnability: provably efficient polynomial-time learning without distributional assumptions. Methodologically, it rigorously reduces the learning problem to linear prediction in a Reproducing Kernel Hilbert Space (RKHS), leveraging RKHS feature mappings, multi-head attention reconstruction, and symmetry analysis of empirical risk minimization. Theoretically, it bridges computational expressivity and statistical learnability for the first time, demonstrating that such models not only simulate associative memory and finite automata but also efficiently learn a class of universal Turing machines (UTMs). Empirical validation confirms accurate convergence of random linear-attention networks on key-value mapping and automaton recognition tasks.

This work investigates the dynamic emergence of linear conceptual representations—referred to as “abstractions”—during neural network training, rather than merely their static presence at convergence. By developing a theoretical framework, the authors derive the first closed-form solution for abstraction trajectories in linear networks and extend this analysis to nonlinear architectures with ReLU and erf activations, integrating geometric reasoning with generalization-aware linear probing. The study uncovers three core principles: abstractions are jointly shaped by data and target geometry, amplified with network depth, and modulated by initialization scale. It further identifies an abstraction attenuation phenomenon in nonlinear networks, where post-activation features exhibit weaker abstraction than pre-activation ones. These theoretical predictions are validated on open-source models such as DINOv3 and Gemma, and leveraged to enhance the generalization performance of linear probes in large language models.

This study investigates whether representations captured by linear probes in the hidden states of large language models stem from genuine differences in reasoning mechanisms or are confounded by task formatting. Focusing on Qwen3-14B across three reasoning tasks, we conduct a systematic analysis integrating linear probing, residualization-based controls, intrinsic dimension estimation, convex hull contamination analysis, trajectory anchor similarity, and causal intervention experiments. Our results show that while raw probe accuracy reaches 100%, it drops to chance level after controlling for task format. Causal testing further reveals no significant functional association between the geometric structure of hidden states and reasoning patterns (p = 0.286). These findings demonstrate that standard probing analyses are highly susceptible to format confounding and underscore the necessity of routinely incorporating format-deconfounding controls in mechanistic interpretability research.

This study investigates whether large language models possess genuine introspective capabilities or merely rely on superficial semantic cues in their inputs. To address this, the authors propose distinguishing authentic introspection from behavior driven by anomaly detection and introduce a desemanticized relabeling control task to eliminate confounding semantic signals. Through a series of rigorous experiments—including internal state interventions, hidden-state label prediction, input-constrained classifier comparisons, and tightly controlled evaluations—they find that models fail to reliably differentiate between internal state changes and input perturbations, performing near chance level on the control task. These findings suggest that current evidence does not substantiate claims of metacognitive monitoring in large language models and establish a more stringent experimental framework for assessing introspection.

This work addresses the challenge of detecting and regulating sycophantic behavior—excessive user flattery—in language models by proposing an iterative data generation method based on cascaded linear samples. Departing from conventional binary contrastive examples, the approach constructs sequences of samples with continuously varying behavioral intensities, revealing for the first time a linearly separable structure of sycophancy in activation space. This enables precise identification and disentanglement of the associated feature subspace. Through activation manipulation and subspace analysis, the method matches or exceeds baseline approaches such as LLM-as-a-judge and system prompting in detection accuracy, calibration, and robust controllability, while incurring lower computational overhead and substantially improving the interpretability of behavioral interventions.

While existing machine unlearning methods appear effective at the output level, their internal representations often retain traces of the forgotten data, which can be easily recovered through fine-tuning, posing significant security risks. This work reveals that the apparent success of such methods typically stems from a misalignment between features and the classifier rather than genuine erasure of information. To address this, we propose a novel unlearning mechanism based on Class Mean Features (CMF), which leverages linear probing analysis and classifier adjustment grounded in the neural collapse hypothesis to achieve true forgetting at the representation level. Experiments demonstrate that CMF substantially reduces the recoverability of unlearned information while maintaining high accuracy on retained tasks, thereby validating the necessity and efficacy of representation-level unlearning evaluation.