V-Gym: Enhancing Agentic Visual Reasoning via Skill-Data Co-Evolution

2026-09-28

AI Summary

This study addresses the unreliability of agent skills arising from limited experience and the lack of targeted practice inherent in static data by proposing an autonomous closed-loop framework. The core innovation lies in introducing a bidirectional co-evolution mechanism between skills and data: targeted multimodal practice data is generated via bottleneck analysis, while hierarchical programmatic skills are dynamically updated based on validation performance. The proposed method integrates trajectory distillation, hierarchical modeling, utility-balanced sampling, and automated verification techniques. Experimental results demonstrate that this framework significantly enhances reasoning performance across multimodal benchmarks, exhibits strong cross-domain generalization, and supports autonomous diagnosis alongside continuous improvement.

Abstract

Advances in multimodal understanding, reasoning, and tool use enable agents to tackle increasingly complex visual reasoning tasks. By distilling past execution experience into reusable skills, agents can transfer lessons from both successes and failures into future reasoning, reducing repeated errors and improving capabilities. However, limited experience may produce unreliable, poorly generalizable skills, while static datasets may lack the targeted and diverse practice needed for refinement. To address this gap, we introduce V-Gym, an autonomous framework that iteratively co-evolves procedural skills and multimodal practice data from execution trajectories. During skill evolution, V-Gym analyzes trajectories to distill and refine hierarchical skills, updating procedural guidance and applicability conditions while retaining an update only if it improves validation performance. During data evolution, V-Gym selects generation seeds by balancing data utility and exploration, then translates trajectory-identified bottlenecks into diverse, targeted practice data that expand the data bank after quality checks. The resulting practice outcomes feed back into subsequent skill updates, closing the loop for continual skill refinement. Experiments across diverse multimodal reasoning benchmarks show substantial improvements over baselines with multiple backbone models. Its evolved skills generalize across domains and models, while evolved data support more effective skill refinement, enabling autonomous diagnosis, targeted practice, and continual self-improvement.

Problem

• Visual Reasoning
• Skill Evolution
• Data Co-Evolution
• Multimodal Agents
• Self-Improvement

Innovation

• Skill-Data Co-Evolution
• Agentic Visual Reasoning
• Autonomous Self-Improvement
• Hierarchical Skills
• Multimodal Practice Data