Next Thoughts Are Distributions: Generative Autoregressive Reasoning in the Latent Space

2026-09-27

AI Summary

This study addresses the challenge of unifying multiple valid reasoning paths within a compact latent space during inference. To this end, it proposes the ATF framework, which models the next thought as a multimodal distribution. By freezing the pretrained backbone and integrating a causal autoregressive model with a lightweight diffusion head, ATF enables continuous reasoning that supports variable step lengths through a reinforcement learning-driven sampling feedback mechanism. Experimental results demonstrate that ATF significantly improves mathematical reasoning accuracy while effectively covering a broader range of solution trajectories. These findings validate the advantages of multimodal prediction in complex reasoning tasks, offering a principled approach to capturing diverse reasoning strategies without compromising representational efficiency.

Abstract

Reasoning problems often admit multiple valid ways to proceed. Continuous reasoning promises to move computation beyond language tokens into a more compact latent space, but representing several plausible ways to think next remains difficult. We introduce Autoregressive Thought Flow (ATF), which models the next continuous thought as a multimodal distribution. A causal autoregressive model performs the reasoning computation, while a lightweight diffusion head generates a plausible next thought from the resulting condition. The sampled thought is fed back into the model, allowing continuous reasoning to unfold for a variable number of steps while preserving the pretrained backbone. Across mathematical reasoning tasks, ATF improves accuracy with compact latent traces and benefits from reinforcement learning and additional test-time thinking. Multi-sample evaluation shows broader solution coverage, indicating that its multimodal predictions capture useful diversity among reasoning paths. Our results suggest that continuous reasoning is more effective when multiple possible next thoughts remain available rather than being collapsed into a single prediction.

Problem

• continuous reasoning
• latent space
• multimodal distribution
• autoregressive reasoning
• reasoning diversity

Innovation

• Autoregressive Thought Flow
• Continuous Reasoning
• Latent Space
• Multimodal Distribution
• Diffusion Head