Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs

2026-09-24

AI Summary

This study addresses the unclear intrinsic mechanisms underlying linear superposition in large language models and demonstrates that pretraining diminishes this property, thereby constraining inference efficiency. We provide the first theoretical proof establishing that linear superposition is an inherent attribute of the Transformer architecture rather than an emergent phenomenon. To exploit this, we propose a lightweight fine-tuning technique to restore model linearity, combined with a guided decoding algorithm that enables multi-stream parallel processing. Our approach significantly reduces the discrepancy between predictive and averaged distributions, successfully generating two coherent text continuations simultaneously within a single forward pass. This work offers a novel paradigm for efficient inference in large-scale models.

Abstract

While Large Language Models (LLMs) rely on highly non-linear components, in this work we demonstrate that they exhibit fundamental linearity: when inputs from distinct text streams are linearly combined, the model outputs a superposition of the individual next-token distributions. We term this the \textit{Superposition Linearity Hypothesis}. We provide evidence that superposition is an intrinsic property of the Transformer architecture rather than an emergent consequence of training; in fact, we observe that it tends to diminish as pretraining progresses. However, we demonstrate that linearity can be substantially restored through lightweight fine-tuning, significantly reducing the divergence between the predicted next-token distribution and the average of the individual next-token distributions. Finally, we introduce a guided decoding procedure that disentangles superposed outputs, enabling the simultaneous generation of two coherent continuations from a single forward pass.

Problem

• Large Language Models
• Linear Superposition
• Transformer Architecture
• Superposition Linearity Hypothesis
• Guided Decoding

Innovation

• Linear Superposition
• Superposition Linearity Hypothesis
• Transformer Architecture
• Lightweight Fine-tuning
• Guided Decoding