LAST: The Last Query Token Guides Visual Token Pruning for Edge-Cloud Collaborative MLLM Inference

2026-07-30

AI Summary

This work addresses the high computational overhead in cloud servers during edge-cloud collaborative multimodal large model inference, caused by dense visual tokens, and the inability of existing pruning methods to achieve query-aware compression prior to transmission. To this end, the authors propose LAST, a novel framework that leverages a lightweight vision-language model on the edge as a proxy agent. LAST exploits the property of causal attention mechanisms—where the last query token has access to the full context—to directly derive importance scores from its attention weights over visual tokens, enabling query-aware pruning without requiring training or access to the cloud model. Combined with a diversity-preserving strategy, LAST accurately selects critical visual tokens under a fixed token budget. Experiments show that LAST retains only 12.5% of visual tokens yet achieves 95.4% of full-token accuracy across 11 benchmarks, substantially reducing edge transmission costs and cloud-side computation load.

Abstract

Multimodal foundation models are reshaping edge-cloud visual intelligence from task-specific feature pipelines into token-based interfaces, where edge devices encode visual inputs into tokens for a general-purpose cloud MLLM. However, dense visual-token sequences increase cloud-side inference costs. Existing pruning methods mainly target centralized inference: vision-driven methods can operate before cloud execution but are typically query-agnostic, whereas query-guided methods often rely on internal states of the target MLLM and cannot determine token relevance before transmission. Compact guidance models offer an alternative, but existing designs may require costly attention aggregation or auxiliary generation. We propose LAST, a training-free framework for query-dependent visual token pruning in edge-cloud collaborative MLLM inference. LAST uses a compact edge-side VLM as a guidance proxy and derives a lightweight importance signal from the last query token's attention to visual tokens. Under causal attention, the last query token can attend to the full visual sequence and the entire query context, enabling query-aware pruning without cloud-model access, autoregressive generation, or costly aggregation over multiple query positions. LAST then retains a diverse set of query-relevant visual tokens under a fixed token budget. We evaluate LAST on 11 multimodal benchmarks under multiple token budgets against pruning methods with different guidance strategies. Experiments show that LAST consistently achieves the strongest performance, preserving 95.4% of the full-token accuracy while retaining only 12.5% of the visual tokens, with low edge-side selection overhead and reduced cloud-side computation.

Problem

• visual token pruning
• edge-cloud collaboration
• multimodal LLM
• query-aware pruning
• inference efficiency

Innovation

• visual token pruning
• edge-cloud collaboration
• query-aware guidance
• training-free framework
• multimodal LLM inference