Problem
Research questions and friction points this paper is trying to address.
Multimodal Large Language Models
Inference Efficiency
Visual Token Compression
Input Resolution
Task-Conditioned Routing
Resolution as a First-Class Decision: Task-Conditioned Routing for Efficient Multimodal Large Language Models
🤖 AI Summary
This study addresses the critical inference bottleneck in multimodal large language models caused by massive visual tokens generated from high-resolution inputs. We propose Task-Conditional Resolution Routing (TCRR), which, for the first time, treats input resolution as a first-class citizen subject to dynamic decision-making rather than a static hyperparameter or one reliant solely on downstream compression. Specifically, TCRR employs a cross-modal router, feature modulation, and cross-attention mechanisms to predict the optimal compression level, supervised by signals generated through a designed teacher oracle pipeline. Evaluated on Qwen3-VL-8B, our method reduces visual FLOPs by 40.9% and latency by 53.7% while maintaining competitive performance.
📝 Abstract
The inference efficiency of Multimodal Large Language Models (MLLMs) is severely constrained by massive visual token sequences induced by high-resolution inputs, with computational cost scaling quadratically. Existing approaches primarily focus on downstream token compression, while overlooking a fundamental upstream inefficiency: input resolution is treated as a static, task-agnostic hyperparameter. We propose Task-Conditioned Resolution Routing (TCRR), which formulates visual compression as a task-conditioned decision and employs a lightweight cross-modal router that conditions backbone visual representations on textual semantics via feature-wise modulation and cross-attention to predict the minimal sufficient compression level per query. To support this, we curate a dataset of 500k samples across 12 task categories, labeled via a teacher-oracle pipeline to approximate Pareto-optimal compression scales. Extensive experiments across diverse architectures show that TCRR achieves a superior efficiency frontier, specifically reducing visual FLOPs by 40.9% and latency by 53.7% on Qwen3-VL-8B while preserving competitive performance. Further analysis of scaling behavior confirms that dynamically routing visual compression enables optimal resource allocation without modifying the MLLM backbone.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Task-Conditioned Routing
Visual Token Compression
Cross-Modal Router
Multimodal Large Language Models
Inference Efficiency
🔎 Similar Papers
No similar papers found.
