OPLD: On-Policy Latent Distillation for Multimodal Reasoning

2026-07-30

AI Summary

Existing vision-language reasoning approaches either rely on predefined reasoning trajectories, limiting flexibility, or perform multimodal alignment only at the feature level without internalizing the reasoning process itself. This work proposes the OPLD framework, which introduces, for the first time, process-level supervision over continuous latent representations. By leveraging an on-policy latent distillation mechanism, OPLD uses multimodal chain-of-thought (CoT) as a teacher signal to guide a student model in reconstructing abstract reasoning dynamics within the latent space. This approach enables the internalization of multimodal reasoning capabilities, significantly outperforming existing latent reasoning methods across multiple benchmarks and achieving state-of-the-art performance, thereby demonstrating the efficacy and superiority of process-level supervision.

Abstract

Interleaved multimodal Chain-of-Thought (CoT) improves visual reasoning by incorporating auxiliary visual evidence into intermediate reasoning. However, existing approaches remain constrained by externally defined reasoning traces and visual operations, limiting their ability to develop flexible and abstract visual thinking. Reasoning with latent has recently offered a promising direction by internalizing intermediate computation into continuous representations. Nevertheless, existing visual-latent methods mainly supervise latent states through alignment with compressed auxiliary visual features, treating them as proxies for visual observations rather than active reasoning states. Consequently, they capture the provided evidence but fail to fully internalize the abstract reasoning process induced by multimodal CoT. In this paper, we propose OPLD (On-Policy Latent Distillation), a simple framework that transfers the reasoning capability induced by privileged multimodal CoT into latent reasoning representations. Extensive experiments on diverse multimodal benchmarks demonstrate that OPLD consistently outperforms existing latent reasoning methods and achieves state-of-the-art performance on multiple benchmarks. The results suggest that supervising latent representations at the reasoning-process level provides a more effective paradigm for multimodal latent reasoning than conventional feature-level alignment.

Problem

• multimodal reasoning
• latent representations
• Chain-of-Thought
• visual reasoning
• reasoning process

Innovation

• On-Policy Latent Distillation
• Multimodal Reasoning
• Chain-of-Thought
• Latent Representation
• Reasoning Process Supervision