HiSC: Hierarchical Spatial Clustering Token Compression for Efficient 3D Scene Understanding

2026-08-05

AI Summary

This work addresses the challenge of visual token inflation in 3D vision-language models, which arises from multi-view redundancy and extensive non-informative regions, rendering existing 2D compression methods inadequate for preserving both 3D structural integrity and fine-grained details. To overcome this, the authors propose HiSC, a novel framework that elevates token compression from individual tokens to cluster-level processing. HiSC constructs spatially anchored token clusters through joint geometric-semantic clustering and introduces a training-free collaborative mechanism comprising Spatial Graph Merging (SGraM) and Hierarchical Clustering Pruning (SCluP). This mechanism eliminates cross-view redundancy prior to LLM inference and dynamically retains critical regions during inference. Evaluated across multiple 3D benchmarks, HiSC achieves over 90% token compression with minimal performance degradation, substantially enhancing inference efficiency.

Abstract

3D vision-language models (3D VLMs) enable spatial reasoning over multi-view scenes but suffer from substantial token redundancy due to duplicated observations and large uninformative regions, leading to high computational cost. Although visual token compression has shown promise in accelerating 2D VLMs, it fails to capture the structured nature of 3D scenes and leads to incomplete spatial coverage and loss of fine-grained details. In this paper, we propose \textbf{HiSC}, a training-free framework for hierarchical spatial clustering token compression in 3D VLMs. HiSC lifts token compression from token-level selection to cluster-level processing by organizing tokens into spatially grounded clusters using joint geometric and semantic cues. Specifically, we first introduce a \textbf{spatial graph-based merging (SGraM) strategy} that models cross-view redundancy as spatial connectivity and consolidates physically consistent regions, effectively merging extremely similar redundant tokens prior to LLM inference. We then propose a \textbf{spatial clustering-based pruning (SCluP) paradigm} within LLM inference, which performs hierarchical compression across clusters and within clusters, preserving object instance completeness while retaining fine-grained details for important regions. Extensive experiments on diverse 3D reasoning benchmarks show validate the effectiveness of HiSC, particularly under high visual token pruning ratios. Besides, HiSC achieves over 90\% token reduction with minimal performance degradation. Code is accessible at https://github.com/elecreak/HiSC.

Problem

• 3D vision-language models
• token redundancy
• spatial reasoning
• visual token compression
• computational cost

Innovation

• hierarchical spatial clustering
• token compression
• 3D vision-language models
• spatial graph-based merging
• spatial clustering-based pruning