Uncovering Latent Reasoning Strategies in Language Models

2026-07-20

AI Summary

This work addresses the challenge that language models implicitly blend multiple reasoning strategies during inference, making them difficult to identify and disentangle. The authors propose a latent variable factorization approach that explicitly decomposes the response distribution into two components: strategy routing and strategy-conditional generation. They introduce a novel variational objective that focuses reconstruction pressure on tokens where a base model exhibits high surprisal while simultaneously maximizing information gain relative to the base model, thereby effectively mitigating posterior collapse. By integrating variational inference with conditional text generation, the method successfully recovers latent codes aligned with reference strategies on multi-strategy algorithmic tasks, achieving interpretability and controllability of reasoning strategies without compromising the original model’s performance.

Abstract

A language model $p_θ(y \mid x)$ trained on reasoning tasks learns to solve problems via multiple distinct strategies, yet these strategies are implicit and entangled within the model's response distribution. We study the problem of decomposing the response distribution of a given pretrained language model into a structured, strategy-conditioned representation. Specifically, we learn a latent-variable factorization $p_θ(y \mid x) \leadsto (r_φ(z \mid x), g_φ(y \mid x,z))$, where a router $r$ maps each input to a distribution over latent strategies $z$ and a generator $g$ produces the response conditioned on that strategy. A key challenge is that the generator, initialized from the base model, already represents $p_θ(y \mid x)$ without using $z$. Standard variational inference therefore gives the model no incentive to route information through $z$ and can yield a severe form of posterior collapse. To address this, we propose a variational objective that measures fractional information gain relative to the base model's response loss and concentrates reconstruction pressure on tokens with high base model surprisal, encouraging $z$ to encode strategy-relevant response variation. We introduce a benchmark of multi-strategy algorithmic tasks and show that this objective recovers latent codes aligned with distinct reference strategies while preserving the base model's response distribution.

Problem

• latent reasoning strategies
• language models
• strategy decomposition
• response distribution
• posterior collapse

Innovation

• latent reasoning strategies
• variational inference
• posterior collapse
• strategy-conditioned generation
• information gain