Technology Category
Computer Vision: Large Vision ModelsMachine Learning: Large Multimodal Models (LMMs)Knowledge Representation and Reasoning: Qualitative Reasoning
Reasoning as Representation: Rethinking Visual Reinforcement Learning in Image Quality Assessment
🤖 AI Summary
Existing reinforcement learning (RL)-based reasoning-style image quality assessment (IQA) models exhibit strong generalization but suffer from opaque mechanisms, high inference energy consumption, and substantial latency.
Method: This paper reveals that their generalization stems from mapping visual representations into compact, cross-domain aligned textual embeddings. Building on this insight, we propose RALI—a lightweight framework that directly aligns image features with such textual representations via contrastive learning, eliminating the need for large language model (LLM) invocation or explicit reasoning steps. RALI integrates RL-guided representation learning, contrastive alignment, and multimodal large model (MLLM)-derived visual features.
Contribution/Results: Experiments demonstrate that RALI achieves state-of-the-art generalization performance on IQA benchmarks—on par with leading reasoning-based models—while reducing parameter count and inference time to less than 5% of theirs, significantly enhancing practical deployability.
Technology Category
Application Category
📝 Abstract
Reasoning-based image quality assessment (IQA) models trained through reinforcement learning (RL) exhibit exceptional generalization, yet the underlying mechanisms and critical factors driving this capability remain underexplored in current research. Moreover, despite their superior performance, these models incur inference energy usage and latency orders of magnitude higher than their earlier counterparts, restricting their deployment in specific scenarios. Through extensive experiments, this paper verifies and elaborates that through RL training, MLLMs leverage their reasoning capability to convert redundant visual representations into compact, cross-domain aligned text representations. This conversion is precisely the source of the generalization exhibited by these reasoning-based IQA models. Building on this fundamental insight, we propose a novel algorithm, RALI, which employs contrastive learning to directly align images with these generalizable text representations learned by RL. This approach eliminates the reliance on reasoning processes and even obviates the need to load an LLM. For the quality scoring task, this framework achieves generalization performance comparable to reasoning-based models while requiring less than 5% of their model parameters and inference time.
Problem
Research questions and friction points this paper is trying to address.
Explores mechanisms behind generalization in reasoning-based IQA models
Addresses high inference energy and latency in visual RL models
Develops efficient alternative to LLM-dependent quality assessment frameworks
Innovation
Methods, ideas, or system contributions that make the work stand out.
Contrastive learning aligns images with text representations
RALI eliminates reasoning processes and LLM loading
Achieves comparable generalization with 5% parameters and time
🔎 Similar Papers
No similar papers found.
