LUT: Latent Utility Training for Visual Reasoning

2026-08-01

AI Summary

This work addresses the reliance of multimodal large language models on costly intermediate supervision for perception-intensive visual reasoning tasks. The authors propose the LUT framework, which optimizes reasoning using only standard VQA data without additional annotations. LUT introduces the novel concept of “latent utility,” employing trajectory-level and step-level utility estimation to guide the model toward generating latent states that are beneficial for the final answer. By integrating utility-aware latent distillation via supervised fine-tuning, answer-attribution-based policy optimization, curriculum learning, and reinforcement learning, LUT significantly outperforms existing implicit reasoning approaches on multiple perception-intensive visual reasoning benchmarks, achieving performance comparable to state-of-the-art methods that require complex interleaved text–latent state annotations.

Abstract

Multimodal large language models have advanced visual understanding, yet perception-intensive reasoning remains challenging. Recent latent visual reasoning methods introduce hidden-space computation before answering, but they often rely on costly intermediate supervision, such as bounding boxes, sketches, or interleaved rationales. These strategies focus on how latent states should be shaped, but do not explicitly assess whether the latent is useful for the final answer. We propose LUT, a latent reasoning framework trained with only standard VQA pairs. LUT centers training on Latent Utility at two levels. At the trajectory level, we propose Utility-Aware Latent Distillation SFT, which explores answer-relevant latent trajectories, selects qualified trajectories by their information gain, and distills more reliable and learnable supervision through curriculum learning. At the step level, we propose Latent Attribution Policy Optimization, which uses answer-to-latent attribution to differentially optimize latent steps during reinforcement learning. Experiments on perception-intensive visual reasoning benchmarks show that LUT outperforms previous latent reasoning methods and remains competitive with latent-text interleaved methods with lower annotation cost.

Problem

• visual reasoning
• latent utility
• multimodal language models
• perception-intensive reasoning
• intermediate supervision

Innovation

• Latent Utility
• Visual Reasoning
• Multimodal LLMs
• Reinforcement Learning
• Curriculum Learning