EvoThink: Evolving Thinking in Large Reasoning Models via Self-Pruning and Aha-Moment Preference Optimization

2026-07-22

AI Summary

This work addresses the challenge of redundant verification steps that often lead large reasoning models into overthinking, a problem inadequately tackled by existing methods which struggle to precisely eliminate redundancy without compromising valid reasoning. To this end, the authors propose Self-Pruning Training (SPT) coupled with an Insight Moment Preference Optimization mechanism. SPT enables fine-grained compression of ineffective reasoning paths, while the latter leverages a genetic algorithm–inspired data synthesis strategy and preference learning to distill efficient reasoning patterns from failed trajectories. This approach uniquely unifies redundancy removal with exploration of novel reasoning pathways, achieving significant reductions in token consumption alongside improved accuracy and generalization on mathematical reasoning and code generation tasks—demonstrating a synergistic gain in both efficiency and performance.

Abstract

Large Reasoning Models (LRMs) often suffer from overthinking due to redundant verification steps. Existing approaches for mitigating overthinking, such as fast-slow thinking switching and reasoning trajectory compression, fail to make a fine-grained distinction between beneficial and redundant steps within the LRM's reasoning process, and may thus impair reasoning capability in their pursuit of efficiency. To simultaneously improve reasoning efficiency and capability, we propose EvoThink, a framework that reduces redundant verification and encourages the exploration of new reasoning paths. EvoThink comprises two key components: Self-Pruning Training (SPT), an unsupervised method that iteratively prunes redundant reasoning steps and self-trains on the concise trajectories; and Aha-Moment Preference Optimization (AMPO), which, inspired by genetic algorithms, identifies valuable failed reasoning attempts, synthesizes from-wrong-to-right aha-moment data, and optimizes the model to internalize this reasoning pattern. Extensive evaluations across mathematical reasoning and code generation benchmarks demonstrate that EvoThink not only substantially reduces inference-time token usage but also improves the reasoning capability of LRMs.

Problem

• overthinking
• Large Reasoning Models
• redundant reasoning steps
• reasoning efficiency
• reasoning capability

Innovation

• Self-Pruning Training
• Aha-Moment Preference Optimization
• Large Reasoning Models
• Reasoning Efficiency
• Genetic Algorithm-inspired Optimization