CIRF: Tokenizing Chain-of-Thoughts into Reusable Functional Units for Efficient Latent Reasoning in Large Language Models

2026-05-27

AI Summary

This work addresses the limitations of existing implicit chain-of-thought (CoT) approaches, which struggle to align with explicit reasoning steps and lack adaptability to input complexity. The authors propose CIRF, a novel framework that, for the first time, decomposes explicit CoT into discrete, interpretable functional semantic units and models them as reusable token sequences. By autoregressively generating these functional tokens along with their optional outcomes, CIRF achieves explicit alignment while enabling parallel training and adaptive handling of reasoning complexity. Experimental results demonstrate that CIRF significantly outperforms current implicit CoT methods across mathematical, symbolic, and commonsense reasoning tasks, achieving a superior trade-off between accuracy and latency, and delivering consistent, interpretable performance gains.

Abstract

Implicit Chain-of-Thought (CoT) reduces the inference cost of large language models by internalizing the explicit rationales. However, existing approaches typically lack alignment with explicit rationales and adaptivity to example complexity. In this work, we propose CIRF (\textit{\underline{C}hain-of-thoughts \underline{I}nto \underline{R}eusable \underline{F}unctional units}), an implicit CoT framework that performs reasoning as a dynamic sequence of discrete functional tokens. CIRF assigns a functional token to each semantically coherent reasoning unit in explicit CoT traces. The model is then fine-tuned to autoregressively generate functional tokens and their optional results, followed by the final answer. This design aligns latent reasoning with a sequence of functional units, facilitating parallel training, explicit rationale alignment, and adaptive reasoning. Extensive experiments on mathematical, symbolic, and commonsense reasoning benchmarks show that CIRF provides a favorable accuracy-latency trade-off compared with state-of-the-art implicit CoT methods. Further analyses demonstrate that CIRF constructs distinct, interpretable functional tokens, leading to consistent performance improvements.

Problem

• Implicit Chain-of-Thought
• reasoning alignment
• adaptive reasoning
• functional units
• latent reasoning

Innovation

• Chain-of-Thought
• Functional Tokens
• Implicit Reasoning
• Latent Reasoning
• Large Language Models