Learn Your Own Thoughts: Abstract Token Curriculum

2026-09-17

AI Summary

本文提出抽象令牌课程（ATC），一种无需直接监督即可激发有效连续中间表示的框架，解决了大型语言模型中链式思维技术需要丰富任务特定数据的问题。

Abstract

Large Language Models (LLMs) have achieved remarkable reasoning capabilities by utilizing chain-of-thought (CoT) as a scratchpad for intermediate stages of thinking. However, CoT techniques require explicit supervision on thinking tokens, which requires rich, task-specific data. In this work, we propose Abstract Token Curriculum (ATC), a novel curriculum learning framework that elicits effective continuous intermediate representations without direct supervision or manual scratchpad design. ATC gradually increases problem complexity through a sequence of distributions, training the model to develop internal abstract ``thoughts'' in the continuous representation space. This paper provides both theoretical and experimental evidence for the benefits of ATC and its advantages over previous methods for training continuous thoughts. Theoretically, we show that for learning parity functions with single-layer softmax attention using ATC, attention naturally focuses on the CoT tokens in the context that provide the ``easiest path'' to predicting the next token. Experimentally, we show ATC's effectiveness on graph reachability and arithmetic learning tasks.

Problem

• Large Language Models
• Chain-of-Thought
• Abstract Token Curriculum

Innovation

• Abstract Token Curriculum
• continuous intermediate representations
• curriculum learning
• chain-of-thought