ORION: Teaching Language Models to Reason Efficiently in the Language of Thought

2025-11-28

AI Summary

To address high latency, low coherence, and computational redundancy in large reasoning models (LRMs) caused by verbose chain-of-thought (CoT) generation, this paper proposes a Mentalese-inspired compressed reasoning framework. It replaces natural-language CoT with concise, symbolic “mentalese” chains and introduces Short-Chain Preference Optimization (SLPO), a reinforcement learning algorithm that dynamically balances brevity and accuracy. Integrated with CoT distillation and structured training, the method achieves 4–16× token compression, 5× lower inference latency, and 7–9× reduced training cost across mathematical reasoning, code generation, and task planning—while retaining 90–98% accuracy and outperforming Claude and GPT-4o. The core contribution is the first integration of symbolic mental representations into LRM reasoning compression, establishing an optimized, compact reasoning paradigm grounded in cognitively inspired formalism.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large Reasoning Models (LRMs) achieve strong performance in mathematics, code generation, and task planning, but their reliance on long chains of verbose "thinking" tokens leads to high latency, redundancy, and incoherent reasoning paths. Inspired by the Language of Thought Hypothesis, which posits that human reasoning operates over a symbolic, compositional mental language called Mentalese, we introduce a framework that trains models to reason in a similarly compact style. Mentalese encodes abstract reasoning as ultra-compressed, structured tokens, enabling models to solve complex problems with far fewer steps. To improve both efficiency and accuracy, we propose SHORTER LENGTH PREFERENCE OPTIMIZATION (SLPO), a reinforcement learning method that rewards concise solutions that stay correct, while still allowing longer reasoning when needed. Applied to Mentalese-aligned models, SLPO yields significantly higher compression rates by enabling concise reasoning that preserves the benefits of detailed thinking without the computational overhead. Across benchmarks including AIME 2024 and 2025, MinervaMath, OlympiadBench, Math500, and AMC, our ORION models produce reasoning traces with 4-16x fewer tokens, achieve up to 5x lower inference latency, and reduce training costs by 7-9x relative to the DeepSeek R1 Distilled model, while maintaining 90-98% of its accuracy. ORION also surpasses Claude and ChatGPT-4o by up to 5% in accuracy while maintaining 2x compression. These results show that Mentalese-style compressed reasoning offers a step toward human-like cognitive efficiency, enabling real-time, cost-effective reasoning without sacrificing accuracy.

Problem

• Reduce reasoning latency and token redundancy in large models
• Improve efficiency and accuracy through compressed symbolic reasoning
• Achieve cost-effective real-time reasoning without sacrificing performance

Innovation

• Training models to reason with ultra-compact structured tokens
• Using reinforcement learning to reward concise and correct solutions
• Achieving high compression and lower latency while maintaining accuracy