Attend to Evidence: Evidence-Anchored Spatial Attention Supervision for Multimodal RLVR

2026-05-29

AI Summary

This work addresses the challenge in multimodal reinforcement learning where reliance solely on outcome-based rewards is prone to linguistic priors, hindering the model’s ability to focus on critical visual evidence. To mitigate this, the authors propose EASE, a novel approach that leverages annotated visual evidence regions as process-level supervision. By introducing an evidence-anchored spatial attention mechanism over high-reward trajectories, EASE generates smooth visual token targets that guide the model to align its attention with relevant evidence regions. Notably, the method requires no additional annotations during inference and consistently enhances performance across perception, hallucination mitigation, visual mathematics, and multimodal reasoning tasks—yielding average gains of 2.5 to 3.1 points on Qwen2.5-VL-7B, Qwen3-VL-4B, and Qwen3-VL-8B—while achieving more precise visual attention localization.

Abstract

Reinforcement learning with verifiable rewards (RLVR) improves vision-language models (VLMs) by optimizing outcome rewards derived from final answers. However, such outcome-only rewards do not tell the model which image regions justify an answer. For questions that require visual grounding, these rewards cannot distinguish responses supported by relevant visual evidence from those produced by language-prior shortcuts or lucky guesses. We introduce EASE (Evidence-Anchored Spatial Attention), which augments multimodal RLVR with visual-evidence process supervision. EASE converts annotated evidence regions into a smoothed visual-token target and uses it to guide response-to-image attention during RL training, but only on high-reward trajectories. The annotations are used solely as privileged training labels, while inference requires only the original image and question. Across Qwen2.5-VL-7B, Qwen3-VL-4B, and Qwen3-VL-8B, EASE raises average scores over DAPO by 2.5 to 3.1 points on perception, hallucination, visual math, and multimodal reasoning benchmarks. Diagnostics and ablations show that EASE better aligns visual attention with annotated evidence regions.

Problem

• Reinforcement Learning with Verifiable Rewards
• Vision-Language Models
• Visual Grounding
• Evidence Anchoring
• Spatial Attention

Innovation

• Evidence-Anchored Attention
• Multimodal RLVR
• Visual Grounding
• Process Supervision
• Spatial Attention