CoBa: Cost-Effective Test-Time Scaling via Compute-Balanced Routing

2026-08-07

AI Summary

This work addresses the challenge of efficiently allocating computational resources during inference under a fixed budget to optimize test-time performance. It formalizes test-time scaling as a dynamic computation allocation problem and introduces a statistically guided routing strategy: within a two-stage verification framework, a lightweight verifier first processes an initial candidate set, and samples exhibiting high uncertainty or potential value are selectively routed to a stronger verification module. The approach integrates parameter-weighted token accounting with bootstrap-based significance testing. Evaluated across multiple mathematical and symbolic reasoning benchmarks, the method achieves a macro accuracy of 85.13% while reducing computational cost by 49.1%–58.9% compared to strong baselines, with negligible performance degradation.

Abstract

Test-time scaling is often implemented by spending more compute along one axis: sampling more solutions, extending a chain of thought, or applying a stronger evaluator. Under a fixed inference budget, these choices compete. This paper formulates test-time reasoning as a compute-allocation problem in which a system must decide whether the next unit of compute should be spent on generation, verification, or stopping. We introduce CoBa, a compute-balanced routing policy that first obtains a small set of candidates, applies cheap verification broadly, and routes uncertain or high-value candidates to stronger verification. On 3,129 example-generator evaluations spanning MATH-500, AIME 2024/2025, AMC 2023, and procedural symbolic reasoning, CoBa-Routed-Strong reaches 85.13% macro accuracy, statistically matching a self-evaluation weighted-voting proxy at 85.20% while using 49.1% fewer parameter-weighted tokens. It also matches best-of-16 majority voting within 0.01 macro-accuracy points while using 58.9% fewer parameter-weighted tokens; paired tests retain a small best-of-16 edge at substantially higher cost. Paired bootstrap tests show significant gains over single-sample decoding, while the remaining gap to the pool oracle exposes headroom for sharper routing. For local reasoning systems, test-time scaling becomes a question of where the next computation is most valuable.

Problem

• test-time scaling
• compute allocation
• reasoning systems
• inference budget
• verification

Innovation

• test-time scaling
• compute-balanced routing
• verification routing
• parameter-weighted tokens
• cost-effective inference