Trinity: Self-Evolving Vision-Language Models with a Self-Verifier

2026-10-03

AI Summary

This study addresses the limitation of existing self-evolving vision-language models (VLMs) that rely on answer consistency as a weak reward, failing to verify the image grounding of questions and the correctness of reference answers. To this end, we propose a unified "ask-solve-verify" architecture that integrates multiple roles within a single model. We introduce an external-annotation-free self-verification mechanism that employs exponential moving average (EMA) to filter grounded questions and rectify erroneous consensus. Combined with self-play curriculum learning, this approach effectively mitigates reward sparsity and noise. Evaluated on Qwen3-VL-8B, our method substantially enhances visual reasoning in mathematics and biology, achieving gains of 8.6 points on the SciVQR biology subset and 12.8 points on MathVerse.

Abstract

Self-evolving vision-language models (VLMs), a form of self-improvement in which a model generates its own training data from unlabeled images, are a promising route toward agents that expand their reasoning capability in an unsupervised manner, without relying on ever-larger annotation budgets. Existing methods pair a Questioner that proposes problems with a Solver that answers them, but reward both roles mainly by agreement among sampled answers. Agreement is a weak proxy for truth: it cannot tell whether a question is grounded in the image, whether the proposed reference answer is right, or whether a confident majority is wrong in the same way. We present Trinity, in which one VLM plays three roles, Questioner, Solver, and Verifier, and the Verifier is a self-verifier: an exponential moving average (EMA) of the policy itself, requiring neither labels nor an external judge. The Verifier screens every generated question for image grounding and answer correctness before it becomes supervision, scores Solver reasoning against the image, and adjudicates disputes between the reference answer and a strong Solver consensus, correcting the reference and penalizing the Questioner when the consensus is right. Trained on images alone, Trinity improves Qwen3-VL-8B on mathematical visual reasoning and on science benchmarks with biology content, for example, +8.6 points on the biology split of SciVQR and +12.8 on MathVerse, and its reward dynamics behave as a healthy self-play curriculum should. These results suggest that a self-evolving multimodal agent can strengthen its scientific reasoning from unlabeled scientific images alone, with the model itself serving as the verifier.

Problem

• self-evolving vision-language models
• unsupervised self-improvement
• agreement-based reward
• self-verification
• visual reasoning

Innovation

• Self-evolving VLM
• Self-verifier
• Exponential moving average
• Unsupervised self-play
• Vision-language reasoning