MPCoT: Reward-Guided Multi-Path Latent Reasoning for Test-Time Scalable Vision-Language-Action

2026-06-04

AI Summary

This work addresses the limited reasoning capability and execution fragility of vision-language-action (VLA) policies in long-horizon, high-uncertainty tasks, which stem from single-step action decoding. To overcome these limitations, the authors propose the MPCoT framework, which introduces the first zero-inference-token, multi-path implicit reasoning mechanism. MPCoT initializes M latent reasoning paths, refines them through K iterations of weight-shared optimization, and softly aggregates actions via a reward-guided path preference objective. Its key innovations include a configurable depth-width reasoning architecture and a path-scoring mechanism that integrates expert action consistency, progress signals from world models or vision-language models (VLMs), and task success feedback. Experiments on the LIBERO and CALVIN benchmarks demonstrate significant performance gains on long-horizon tasks, while ablation studies validate the effectiveness of the reasoning structure, confidence-weighted aggregation, and reward-guided supervision.

Abstract

Vision-Language-Action (VLA) policies remain brittle in long-horizon and high-uncertainty control, where one-pass action decoding provides limited inference-time deliberation. Explicit chain-of-thought can increase reasoning depth, but introduces token latency and an indirect text-to-action interface. We propose MPCoT, a reward-guided multi-path latent reasoning framework that initializes $M$ hypotheses, refines them for K weight-tied steps, and softly aggregates them before action decoding. A training-only path-preference objective evaluates candidate action branches with expert-action consistency, world-model/VLM-based progress, and success feedback to align the latent path scorer with downstream execution quality. MPCoT preserves the original 8-step action interface, generates zero reasoning tokens, and exposes configurable inference controls (K,M). Under matched protocols on LIBERO and CALVIN, MPCoT improves long-horizon performance, with ablations confirming depth-width effects, confidence-weighted aggregation, and reward-guided path supervision.

Problem

• Vision-Language-Action
• long-horizon control
• high-uncertainty
• inference-time deliberation
• action decoding

Innovation

• multi-path reasoning
• reward-guided supervision
• latent reasoning
• test-time scaling
• vision-language-action