RouterVLA: Turning Smoke Tests into Supervision for Heterogeneous VLA Selection

2026-06-25

AI Summary

This work addresses the challenge of effectively selecting among heterogeneous vision-language-action (VLA) policies using pre-deployment smoke-test data, avoiding suboptimal performance caused by reliance on a single globally optimal policy. The authors propose RouterVLA, which leverages an outcome-disjoint cross-fitting mechanism to convert smoke-test replays into supervision signals for frozen expert policies and employs a transparent success-rate-based routing rule. Evaluated on LIBERO-Plus using 34,752 replay trajectories, the method improves holdout test-set success rates from 0.4686 to 0.6149 (+14.64%). The study further reveals that learned scorers offer no additional benefit under scalar profiling and that reusing scoring trials inflates estimated gains by up to 1.87×, thereby underscoring the intrinsic value of task assignment itself.

Abstract

We study whether pre-deployment evaluation rollouts can be reused to supervise policy selection. Robot teams routinely smoke test candidate vision-language-action (VLA) policies, then compress those trials into a global winner. RouterVLA evaluates this idea with outcome-disjoint cross-fitting: recorded probes build a profile for each frozen expert, and a separate trial scores the selected expert without entering its profile. Across 34,752 LIBERO-Plus rollout records, a transparent probe-success rule raises held-out success from 0.4686 to 0.6149, a +14.64pp gain. Under the scalar-only profiles studied here, learned scorers are statistically indistinguishable from this rule, showing that commissioning carries the routing value while extra scalar scorer capacity does not create it. Reusing the scored trial inflates the measured gain by $1.87\times$, so credible ledger routing needs outcome separation; model scaling improves individual policies, while commissioning-aware routing improves the system built from them.

Problem

• VLA selection
• smoke tests
• policy routing
• pre-deployment evaluation
• robot policy

Innovation

• RouterVLA
• outcome-disjoint cross-fitting
• VLA policy selection
• smoke test reuse
• commissioning-aware routing