MedARC: Training-Free Adaptive Redundancy Compression of Visual Tokens for 3D Medical Vision-Language Models

2026-07-29

AI Summary

This work addresses the challenge of excessive and highly redundant visual token sequences generated when feeding 3D medical images into vision-language models (VLMs), where existing compression methods often discard clinically critical information. The authors propose a training-free, adaptive token compression framework that, for the first time in 3D medical VLMs, integrates three complementary cues—self-attention from the visual encoder, image-text semantic alignment, and local anatomical saliency—to dynamically assess token importance. A saliency-aware merging strategy is further designed to preserve diagnostically essential regions. Evaluated on the CT-RATE and MR-RATE datasets, the method substantially reduces token count and inference time while maintaining or even improving diagnostic performance, demonstrating particular efficacy in large language model settings.

Abstract

Integrating 3D medical images with vision-language models (VLMs) holds substantial promise for computer-aided diagnosis. However, volumetric images generate prohibitively long visual-token sequences with considerable spatial and inter-slice redundancy. Existing token compression methods typically apply uniform reduction or rely on a single importance signal, increasing the risk of removing regions that are clinically relevant to the query or structurally distinctive. To address this limitation, we propose MedARC, a unified, training-free framework for Adaptive Redundancy Compression of visual tokens in 3D medical VLMs. MedARC estimates token importance by integrating three complementary cues: self-attention from the VLM vision encoder, which reflects the model's intrinsic visual focus; similarity between projected visual tokens and text embeddings, which identifies query-relevant regions; and deviations of local visual foundation model features from the volume-level feature center, which highlight structurally distinctive anatomy. The resulting importance distribution guides a saliency-aware merging strategy that preserves informative tokens while consolidating redundant ones rather than simply discarding them. Experiments on CT-RATE and MR-RATE show that MedARC reduces visual-token overhead and inference time while preserving or improving diagnostic performance. Its multi-cue scoring cost is outweighed by the savings from processing fewer tokens, with greater benefits expected for larger language models.

Problem

• 3D medical imaging
• vision-language models
• token compression
• redundancy
• clinical relevance

Innovation

• adaptive token compression
• training-free
• multi-cue importance estimation
• 3D medical vision-language models
• saliency-aware merging