Recursive Harness Self-Improvement for Frontier Reasoning Data Synthesis

2026-10-02

AI Summary

This study addresses the insufficient task difficulty progression and static generation frameworks prevalent in existing reasoning data synthesis by proposing a task-framework co-evolution paradigm. The method integrates dual self-improvement mechanisms—online and post-processing—to dynamically optimize the generation pipeline through recursive self-improvement, skill reuse, and automated revision of prompts and workflows. This approach effectively overcomes the difficulty stagnation bottleneck under fixed model weights and evaluation criteria. Experimental results demonstrate that the proposed framework reduces solver accuracy to 54.8%, thereby significantly enhancing downstream supervised fine-tuning (SFT) and Group Relative Policy Optimization (GRPO) performance. Notably, a 27B-parameter model achieves 62.5% accuracy on the APEX benchmark, validating the efficacy of the co-evolutionary approach for generating high-quality synthetic reasoning data.

Abstract

Generating progressively harder reasoning problems requires synthesis procedures that adapt as the task distribution evolves. Existing task-level recursion reuses generated problems as seeds but leaves the construction harness unchanged. We present task-harness co-evolution, a framework for recursive harness self-improvement (RSI) in reasoning-data synthesis. Online self-improvement converts intermediate solver failures into reusable skills during generation. Post-task self-improvement revises skills, prompts, and workflows after each batch, adopting candidates only when they generate harder valid tasks within a bounded cost increase. Model weights and verification criteria remain fixed. Across mathematics, coding, and science, mean solver accuracy decreases from 100.0% to 54.8% over fourteen evolution rounds. Ablations show that combining both update schedules produces harder tasks than fixed-harness recursion or either schedule alone. The resulting data improves downstream SFT and GRPO performance. In particular, a 27B student fine-tuned on 10K synthesized mathematics examples achieves 62.5% mean-16 accuracy on APEX, competitive with selected frontier-model references. These results support adapting the synthesis harness alongside the tasks to generate increasingly challenging data with downstream training value.

Problem

• reasoning data synthesis
• recursive self-improvement
• task-harness co-evolution
• frontier reasoning

Innovation

• Recursive Harness Self-Improvement
• Task-Harness Co-evolution
• Reasoning Data Synthesis
• Online Self-Improvement
• GRPO