J-CoT: Chain-of-Thought in J-Space

2026-07-24

AI Summary

This work addresses the inefficiency of existing chain-of-thought (CoT) methods, which rely on verbose natural language intermediate steps, and the lack of explicit information selection in implicit reasoning approaches that operate over dense hidden states. To overcome these limitations, the authors propose the J-CoT framework, which constructs a vocabulary-indexed J-space within hidden representations. At each reasoning step, the model encodes its intermediate state as a sparse set of lexical index coefficients—termed J-thoughts—and passes them to subsequent steps, thereby avoiding the generation of full sentences or transmission of dense vectors. This structured, interpretable interface balances linguistic readability with computational efficiency. Experiments demonstrate that, under identical backbone models and settings, J-CoT-Zero matches or exceeds the strongest implicit baselines across all benchmarks, while J-CoT-Train achieves state-of-the-art performance on tasks involving mathematical, scientific, programming, and structured path reasoning.

Abstract

Chain-of-thought prompting improves language-model reasoning by carrying intermediate states across successive computation steps. However, relying on natural language as the only recurrent interface is overly restrictive, since many transient computations do not need to be fully verbalized. Existing latent-reasoning methods remove this constraint by recurrently propagating continuous hidden states. However, these methods pass a dense hidden vector as a whole, without an explicit mechanism for selecting and organizing the information needed by the next reasoning step. This motivates an intermediate interface that remains linguistically grounded without requiring a decoded sentence. We introduce \textbf{J-CoT}, a recurrent reasoning framework built on \emph{J-space}, a vocabulary-indexed coordinate system within the model's hidden representations. Within each cycle, the model computes in its full hidden space. At the cycle boundary, J-CoT expresses the intermediate state as vocabulary-indexed coefficients, carries these coefficients forward as a \emph{J-thought}, and maps them back into the model's hidden representation for the next cycle. J-CoT therefore requires neither a fluent intermediate rationale nor recurrence over the complete hidden state. Under matched backbone and inference settings, J-CoT-Zero matches or exceeds the strongest evaluated latent-reasoning baseline on every benchmark, while J-CoT-Train obtains the highest score across the evaluated mathematical, scientific, coding, and structured path-reasoning tasks.

Problem

• Chain-of-Thought
• Latent Reasoning
• Intermediate Representation
• Language Models
• Reasoning Efficiency

Innovation

• J-CoT
• J-space
• chain-of-thought
• latent reasoning
• vocabulary-indexed representation