🤖 AI Summary
This paper addresses the challenge of *concept-existence–driven dynamic triggering*—i.e., conditioning large language model (LLM) outputs on the presence or absence of specific concepts during generation. We propose the Logic-Entailment Guided (LEG) mechanism, the first approach to embed neuro-symbolic logic into pretrained Transformers without fine-tuning. Methodologically, LEG constructs interpretable concept vectors in activation space under the linear representation hypothesis and implements neural-symbolic steering via conditional logical gating. Our contributions are threefold: (1) it provides human-constructible, formally verifiable reasoning capabilities; (2) it enables high-precision, attributable, and invertible concept-conditioned responses; and (3) it significantly improves controllability and interpretability across diverse generative tasks—including controlled text generation, concept editing, and safety-aligned output steering—while preserving base-model fidelity.
📝 Abstract
The field of mechanistic interpretability in pre-trained transformer models has demonstrated substantial evidence supporting the ''linear representation hypothesis'', which is the idea that high level concepts are encoded as vectors in the space of activations of a model. Studies also show that model generation behavior can be steered toward a given concept by adding the concept's vector to the corresponding activations. We show how to leverage these properties to build a form of logical implication into models, enabling transparent and interpretable adjustments that induce a chosen generation behavior in response to the presence of any given concept. Our method, Logical Implication Model Steering (LIMS), unlocks new hand engineered reasoning capabilities by integrating neuro-symbolic logic into pre-trained transformer models.