Collaborative Reasoning Distillation via Cross-Feedback and Coherent Curation

2026-10-07

AI Summary

This study addresses the high computational costs of reasoning distillation for large language models and the difficulty in distinguishing valid reasoning from fortuitous guessing. To this end, it proposes a Collaborative Reasoning Distillation (CRD) framework alongside a budget-constrained Reasoning Quality Optimization (RQO) algorithm. Methodologically, CRD introduces an interactive cross-feedback mechanism among teacher models, incorporates fine-grained step-level logical validity assessment, and achieves coherent step splicing through complementary advantage synthesis, thereby systematically enhancing the reasoning capabilities of smaller models. Experimental results demonstrate that, using only 50,000 training samples, the CRD-4B model achieves 97.3% on MATH-500 and 70.3% on AIME'25, significantly outperforming existing baselines.

Abstract

Reasoning capabilities are critical for advancing Large Language Models, yet current approaches either require massive computational budgets or struggle to effectively distill reasoning to smaller models. Standard distillation methods rely on outcome-based rewards, failing to distinguish between sound reasoning and lucky guesses. We propose Collaborative Reasoning Distillation (CRD), a framework that enhances reasoning in compact models through three innovations: (1) interactive cross-feedback where teachers iteratively critique each other's reasoning, (2) fine-grained step-wise quality assessment capturing logical validity independent of final answers, and (3) coherence-aware step stitching that synthesizes complementary strengths. Students are trained via Reasoning Quality Optimization (RQO) with budget constraints. Our model, CRD-4B, achieves 97.3% on MATH-500 and 70.3% on AIME'25, surpassing baselines while using only 50K training examples, up to 12 times smaller than the datasets of comparable models.

Problem

• reasoning distillation
• large language models
• knowledge distillation
• outcome-based rewards
• computational budget

Innovation

• Collaborative Reasoning Distillation
• Cross-Feedback
• Step-wise Quality Assessment
• Coherence-aware Step Stitching
• Reasoning Quality Optimization