OPOD: On-Policy Omni Distillation

2026-07-23

AI Summary

Unified multimodal models often underperform specialized counterparts due to imbalanced learning across modalities. This work proposes a modality-aware teacher routing mechanism that directs student outputs to modality-specific teachers and introduces token-level confidence filtering to distill only the knowledge for which teachers exhibit higher confidence than the student. Additionally, distillation weights are independently tuned per modality, and the model is jointly optimized for both answer correctness and reasoning plausibility. Evaluated across 12 benchmarks and three model scales, the approach achieves state-of-the-art average performance; notably, the 30B variant surpasses all baselines and jointly post-trained models, ranking among the top two on 11 tasks. These results demonstrate that a single general-purpose model can deliver highly effective multimodal capabilities without requiring multiple specialized systems.

Abstract

Omni-modal models can handle text, images, and audio in one system, but improving all of these abilities together remains difficult. Training a single model on pooled multimodal data often fails to match models specialized for individual modalities. On-policy distillation (OPD) offers a way to combine such specialists: the student generates a response, and a teacher evaluates that same response, so the student learns directly from behaviors it actually produces. Yet using several teachers can introduce competing guidance and improve one modality at the expense of another. We present On-Policy Omni Distillation (OPOD), which routes each student response to the matching text, image, or audio teacher. OPOD keeps teacher guidance only on tokens where the teacher assigns a higher probability than the student, adjusts the influence of each modality teacher independently during training, and asks the routed teacher to assess both the final answer and whether the reasoning supports the correct answer. Across twelve benchmarks and three backbone sizes, OPOD achieves the best average score at every scale, reaching 70.8, 51.7, and 46.2 and exceeding the strongest comparator by 2.1, 1.8, and 1.7 points. On the 30B model, it outperforms both the base model and a counterpart post-trained jointly on pooled multimodal data on all twelve benchmarks, and ranks first or second on eleven even when the individual specialists are included. The specialists are discarded after training, leaving one deployable omni-modal model. These results show that coordinating modality-specific teachers is an effective way to improve a shared model while maintaining cross-modal balance.

Problem

• omni-modal models
• multimodal learning
• modality balance
• on-policy distillation
• cross-modal interference

Innovation

• On-Policy Distillation
• Omni-modal Learning
• Modality-specific Teachers
• Dynamic Routing
• Cross-modal Balance