Self-Boosting Vision-Language Models with Noisy Student On-Policy Self-Distillation

2026-07-25

AI Summary

Existing vision-language models rely on human annotations, external distillation, or human feedback for post-training, making it challenging to achieve continuous improvement without external supervision. This work proposes NOPD, a novel approach that, for the first time, leverages prediction inconsistencies induced by input perturbations as a self-supervised signal. By integrating a Noisy Student framework with an on-policy self-distillation mechanism, NOPD uses the model’s predictions on clean inputs to guide its learning from corrupted inputs at the token level—eliminating the need for external models or labels. The method matches or surpasses reinforcement learning and distillation-based approaches across five visual reasoning tasks: it achieves a 20-point gain on the Geometry3K validation set with only 2.1K samples and a 7.4-point improvement on MathVista, demonstrating consistent effectiveness across three mainstream architectures on twelve benchmarks.

Abstract

Post-training enables vision-language models (VLMs) to understand human instructions and perform various downstream tasks. Current post-training methods usually rely on human-annotated data, distillation from external models, reinforcement learning with human feedback, or verifiable answers. This limits their ability to improve without external supervision. To tackle this, we propose NOPD (Noisy Student On-Policy Self-Distillation), a simple yet effective self-distillation approach that improves VLMs without any external models or ground-truth answers. Our key insight is that prediction discrepancies between clean and corrupted inputs naturally induce a self-supervision signal. In NOPD, the model learns from corrupted inputs while using its own predictions under clean inputs as token-level supervision. We show the effectiveness of NOPD on five visual reasoning tasks; it can match and even outperform reinforcement learning approaches or distillation from external models. Notably, when trained with 2.1K samples from Geometry3K, NOPD improves Qwen2.5-VL-7B by 20 points on its validation set. It also shows generalization on out-of-distribution test sets and achieves 7.4 point gains on MathVista. Furthermore, we demonstrate that NOPD is a general approach to enhance VLMs, achieving improvements across three models on 12 benchmarks.

Problem

• vision-language models
• post-training
• external supervision
• self-distillation
• visual reasoning

Innovation

• self-distillation
• vision-language models
• noisy student
• on-policy learning
• self-supervision