Better Starts, Better Ends: Bootstrapped Iterative Self-Reasoning Distillation for Compressed Reasoning

2026-07-17

AI Summary

This work addresses the inefficiency of large reasoning models that often generate redundant or off-topic reasoning chains, consuming excessive computational resources without consistent performance gains. To mitigate this, the authors propose BIRD, a two-stage self-reasoning distillation framework that refines the model’s reasoning path distribution. First, concise instructions are used to sample correct and succinct reasoning trajectories, which are then leveraged for supervised fine-tuning with prompt switching. Subsequently, an intra-policy inverse KL distillation mechanism internalizes this instruction-induced conciseness as the model’s default behavior, overcoming the initialization limitations of conventional intra-policy distillation and substantially improving distillation efficiency. Evaluated on Qwen3-8B, BIRD boosts accuracy on MATH-500 from 86.2% to 92.0% while reducing average response length from 3,099 to 1,115 tokens, achieving superior accuracy–efficiency trade-offs on both MATH-500 and AIME benchmarks.

Abstract

Large reasoning models often solve problems through long chain-of-thought (CoT) traces, yet much of this computation is spent on redundant derivations, repeated self-verification, and detours that do not improve the final answer. Existing on-policy self-distillation methods reduce this cost by matching a student model to a concise copy of itself on prefixes sampled from the student's own rollouts. We show that this objective has an initialization bottleneck. Since supervision is applied only to visited prefixes, training from a verbose base model places the KL loss on contexts that are often noisy, redundant, or already off track. In such regions, a concise teacher can provide only local corrections, while the student continues to explore trajectories that an efficient reasoner should avoid. In this paper, we propose BIRD(Bootstrapped Iterative Self-Reasoning Distillation), a two-stage self-reasoning distillation method that improves the rollout distribution before on-policy training. BIRD first samples concise solutions from the base model under a brevity instruction, keeps only answer-correct traces, and performs a lightweight prompt-switch SFT step. The traces are generated with the brevity instruction but learned under the original task prompt, turning instruction-induced conciseness into a default reasoning behavior. Starting from this warm model, BIRD then applies on-policy reverse-KL distillation with a concise self-teacher, now on cleaner and more informative prefixes. Across Qwen3 series models, BIRD achieves a stronger accuracy-efficiency trade-off than prompting and cold-start on-policy distillation on MATH-500 and AIME benchmarks. On Qwen3-8B, it improves MATH-500 accuracy from 86.2% to 92.0% while reducing the average response length from 3,099 to 1,115 tokens. These results highlight prefix support as a central factor in efficient reasoning distillation.

Problem

• chain-of-thought
• self-distillation
• reasoning efficiency
• prefix support
• model compression

Innovation

• self-distillation
• chain-of-thought compression
• reasoning efficiency
• prefix-aware training
• bootstrapped reasoning