Hi-Token: Hierarchical Coordinate Tokenization for Generative Visual Grounding

2026-08-04

AI Summary

This work addresses the limitation of existing generative vision-language models that treat bounding box coordinates as unordered symbols, disregarding their numerical order and axis-specific semantics, which leads to imprecise localization. To overcome this, the authors propose Hi-Token, a hierarchical coordinate tokenization scheme that decomposes each coordinate into hundreds, tens, and units digits using axis-aware tokens. They further introduce Hi-GAR, a geometry-based multi-scale reward mechanism, optimized via Group Relative Policy Optimization (GRPO). Notably, this approach requires no architectural modifications to the underlying model. Evaluated on the RefCOCO benchmark suite, Hi-Token consistently improves performance across three mainstream vision-language model backbones, with Hi-R1 outperforming strong baselines on most metrics, significantly reducing low-overlap predictions and enhancing localization accuracy.

Abstract

Generative Vision-Language Models (VLMs) commonly treat bounding-box coordinates as independent output symbols, leaving numerical order and axis semantics implicit. We identify this representation as an important source of error in visual grounding. Hi-Token encodes each coordinate with axis-specific tokens for the hundreds, tens, and ones digits, which adds coarse-to-fine structure and increases token reuse while retaining the existing VLM architecture. Hi-GAR complements this representation with a geometry-based reward for Group Relative Policy Optimization (GRPO), using box overlap and coordinate accuracy at multiple scales. Controlled comparisons under matched training conditions show that Hi-Token improves localization throughout the evaluated IoU range. Hi-GAR further reduces low-overlap predictions and is used only during training. Experiments on three VLM backbones and the RefCOCO family show consistent gains across models and benchmarks. Hi-R1 achieves higher values than strong specialist baselines on most reported metrics. Analyses of token frequency, digit boundaries, object scale, and IoU distributions explain the effects of coordinate representation and reward training. The results show that structured coordinate generation provides an effective approach to generative visual grounding.

Problem

• visual grounding
• coordinate representation
• generative vision-language models
• bounding-box coordinates
• numerical semantics

Innovation

• Hi-Token
• hierarchical tokenization
• generative visual grounding
• coordinate representation
• geometry-based reward