Ordered Action Tokens for Visuomotor Policy Learning

2026-07-23

AI Summary

Existing action tokenization methods struggle to simultaneously achieve compact sequence length, structured representation, and compatibility with downstream policies. This work proposes Ordered Action Tokenization (OAT), the first approach to jointly attain high compression ratio, full invertibility, and an ordered discrete token space. OAT integrates a register-augmented Transformer, Finite Scalar Quantization (FSQ), and a ranking-aware training mechanism to compress continuous action chunks into progressively decodable, ordered token sequences. This design enables flexible trade-offs during inference between computational cost and action fidelity. Evaluated across more than 60 simulated and real-world tasks, OAT consistently enhances both performance and inference flexibility across diverse policy architectures.

Abstract

Action tokenization maps continuous robot action chunks to discrete tokens and has become an important interface for modern visuomotor policies. Existing approaches either rely on analytical discretization methods that produce prohibitively long token sequences or learned latent tokenizers that lack structure, limiting their compatibility with downstream policies. In this work, we identify three desiderata for action tokenization - high compression, total decodability, and an ordered token space - and introduce Ordered Action Tokenization (OAT), a learned action tokenizer that satisfies all three. OAT discretizes action chunks into an ordered sequence of tokens using a transformer with registers, finite scalar quantization, and ordering-inducing training mechanisms. By training each token prefix to decode into a valid action chunk, OAT places coarse control information in early tokens and uses later tokens to refine residual detail, yielding an anytime tradeoff between inference cost and action fidelity. We validate OAT in two prevailing uses of action tokens: autoregressive policies that generate tokens for control, and token co-training policies that use token losses to shape the vision-language model context consumed by a flow-based action expert. Across three policy backbones and more than 60 tasks spanning five simulation benchmarks and real-world settings, OAT consistently delivers strong policy performance while offering significantly greater flexibility at inference time.

Problem

• action tokenization
• visuomotor policy
• discrete tokens
• policy compatibility
• sequence compression

Innovation

• Ordered Action Tokenization
• action tokenization
• visuomotor policy
• anytime inference
• transformer with registers