SPLIT-RL: Staged Perception-Language Reasoning Training with Claim-Level Advantages

2026-10-07

AI Summary

This study addresses the bottleneck in visual-language reasoning where monolithic reinforcement learning fails to disentangle perceptual errors from linguistic ones. To overcome this limitation, we propose a staged post-training framework that decouples complex reasoning through atomic visual claim decomposition. By introducing a claim-level advantage algorithm integrated with Group Relative Policy Optimization (GRPO), our approach enables fine-grained error isolation and independent optimization of both visual perception and language reasoning capabilities. Extensive experiments demonstrate that the proposed framework yields consistent accuracy improvements ranging from 1.4 to 6.1 percentage points across multiple backbone models. These results confirm that disentangling perception from reasoning during training effectively enhances the dual visual-linguistic reasoning proficiency of vision-language models.

Abstract

Vision-Language (VL) reasoning requires a model to both extract relevant and accurate information from an image (visual reasoning, VR), and to infer the answer from it (language reasoning, LR). Reinforcement learning with verifiable rewards typically trains both through a single chain-of-thought with a final-answer reward. This gives every CoT token the same sequence-level advantage, failing to distinguish capability specific errors. We propose SPLIT-RL, a staged post-training approach that trains VR and LR in disjoint phases. Because a group's rollouts differ along one capability at a time, the group-relative advantage isolates it, and each phase is optimized using phase-specific reward. We further introduce Claim-Level Advantage (CLA-GRPO), which decomposes VR-phase rollouts into atomic visual claims and provides a fine-grained advantage at claim level based on visual-type group formation. Although trained in two phases, trained policy is evaluated like GRPO model, with a single CoT call at inference time. Under this protocol, SPLIT-RL improves average accuracy over GRPO by 1.4-6.1 points across Qwen3-VL models from 2B to 30B-A3B and InternVL3.5-8B. Evaluating each capability using an oracle based diagnostic shows that answer-only GRPO leaves perception unchanged, whereas SPLIT-RL improves both VR and LR.

Problem

• Vision-Language reasoning
• Reinforcement learning
• Visual reasoning
• Language reasoning
• Sequence-level advantage

Innovation

• Staged Training
• Vision-Language Reasoning
• Claim-Level Advantage
• Reinforcement Learning
• GRPO