Refining Over Resampling: Test-Time Self-Correction for LLM Reasoning

2026-08-06

AI Summary

This work addresses the limitations of existing test-time scaling methods, which often suffer from insufficient sampling diversity or reliance on external verifiers. The authors propose a verifier-free breadth–depth optimization framework that, during inference, iteratively performs self-critique and refinement across multiple reasoning trajectories, followed by majority voting to aggregate final answers. By eliminating dependence on external reward models, the approach effectively balances broad exploration with deep reasoning, significantly enhancing both computational efficiency and accuracy. Empirical results demonstrate substantial improvements over greedy decoding, conventional majority voting, and state-of-the-art verifier-based sampling strategies on mathematical reasoning benchmarks such as MATH500 and AMC; for instance, Qwen2.5-1.5B achieves an accuracy of 32.5% on AMC, up from 25.0%.

Abstract

Test-time scaling improves LLM reasoning by using additional inference compute, but wider sampling alone can suffer from diminishing returns: new rollouts often repeat existing answer patterns instead of adding useful reasoning diversity. Verifier-based selection offers an alternative, but its performance depends on the calibration of an external reward model. We propose a verifier-free breadth--depth refinement framework that uses test-time compute to both explore and improve candidate solutions. The method samples multiple independent reasoning rollouts, refines each rollout through iterative self-critique and self-correction, and aggregates the refined answers by majority voting. Breadth preserves diverse initial attempts, while depth repairs local reasoning errors before aggregation. Across AIME24, AIME25, AMC, OlympiadBench, and MATH500, our method consistently improves over greedy decoding, majority voting, verifier-based best-of-$N$, beam search, and lookahead decoding across multiple open-weight models. For instance, with Qwen2.5-1.5B, accuracy increases from the strongest verifier-based baseline to $58.0\%$ on MATH500, and from $25.0\%$ to $32.5\%$ on AMC. These results show that test-time compute can be more effective when used to refine sampled trajectories rather than only to sample more candidates or rely on verifier-guided selection.

Problem

• test-time scaling
• reasoning diversity
• verifier calibration
• diminishing returns
• LLM reasoning

Innovation

• test-time self-correction
• verifier-free refinement
• breadth-depth reasoning
• iterative self-critique
• majority voting aggregation