CALR: Continuous Anchored Latent Reasoning via Render-of-Thought Compression

2026-10-05

AI Summary

This study addresses semantic collapse in continuous latent variable reasoning and the tendency of discrete methods to bypass intermediate states by proposing a Continuous Anchored Latent Reasoning framework. The approach introduces an information-balanced compression of reference latent variables and employs a functional anchoring mechanism to bridge latent variable formation with answer generation. By integrating derivation-level semantic anchoring with a parallel-to-autoregressive curriculum learning strategy, the framework achieves efficient visual reasoning compression. Experimental results demonstrate that this method significantly improves accuracy across five mathematical benchmarks, outperforming comparable continuous approaches by 26.0 percentage points while effectively preserving intermediate reasoning capabilities.

Abstract

Visual latent reasoning compresses rendered derivations into compact intermediate states, reducing textual reasoning overhead. Existing approaches differ in how they represent these states: continuous methods avoid vocabulary constraints, whereas discrete methods improve accuracy through quantization into a finite codebook. Our analysis of representative continuous and discrete systems identifies two functional requirements: answers must rely on latent states, and those states must carry valid, problem-specific reasoning. Continuous latents influence answers despite collapsed reasoning content, whereas discrete latents retain recoverable intermediate reasoning that answer prediction largely bypasses. To address these challenges, we propose Continuous Anchored Latent Reasoning (CALR), which connects latent formation with answer use through functional anchoring. With reference latents from information-balanced compression, CALR couples latent-mediated answer supervision with derivation-level semantic anchoring: the former routes answer supervision through intermediate states, while the latter grounds their decoded content in problem-specific derivations. A parallel-to-autoregressive curriculum develops sequential reasoning by conditioning subsequent latent blocks on generated prefixes. Evaluations on five mathematical reasoning benchmarks across model families show substantial accuracy gains. Under matched budgets, CALR gains 26.0 percentage points over a comparable continuous latent reasoning method. Further analyses show that its latents support answer prediction and carry problem-specific intermediate reasoning.

Problem

• visual latent reasoning
• continuous latent representation
• discrete latent representation
• reasoning collapse
• mathematical reasoning

Innovation

• Continuous Anchored Latent Reasoning
• Render-of-Thought Compression
• Functional Anchoring
• Semantic Anchoring
• Parallel-to-Autoregressive Curriculum