Direction-Scale Decomposition in Action Representation: Rethinking What to Tokenize for Vision-Language-Action Models

2026-09-23

AI Summary

This study addresses the sensitivity of conventional action representations to execution speed, which often obscures underlying geometric structures. To this end, we propose Direction-Scale Decomposition (DSD), a method that decouples translation and rotation into directional and scalar components prior to tokenization. By isolating motion directions while preserving magnitude information, DSD enhances cross-dataset generalization. We further construct a comprehensive representation framework by integrating uniform binning (BIN) with a B-spline tokenizer (BEAST). Experimental results demonstrate that the proposed approach significantly improves success rates on the LIBERO benchmark and yields a 10.3% performance gain on SimplerEnv under mixed-training settings. The effectiveness of the framework is additionally validated through real-world robot experiments.

Abstract

Action representation plays a central role in discrete-token vision-language-action (VLA) learning but remains underexamined. Under conventional pose-increment representations, action tokens are sensitive to execution speed and dataset-specific normalization, potentially obscuring geometric structure shared across demonstrations and datasets. We introduce Direction-Scale Decomposition (DSD), an action representation that decomposes translation and rotation increments into direction and scale components before tokenization. DSD isolates motion direction while retaining magnitudes in separate scale channels. We evaluate DSD with uniform binning (BIN) and BEAST, a B-spline-based tokenizer, in simulation and real-world manipulation under both single-dataset and mixed-dataset training. On LIBERO, DSD improves average success rates with both tokenizers. On SimplerEnv, DSD-BIN outperforms BIN by 10.3 percentage points in overall success rate under mixed-dataset training. Real-robot experiments further show gains both with and without robotics pretraining. These results support DSD as an effective action representation for discrete-token VLA models and suggest its potential to mitigate performance degradation when training on large and diverse dataset mixtures. Our project page with additional resources is available at https://vla-dsd.github.io/

Problem

• Vision-Language-Action Models
• Action Representation
• Discrete Tokenization
• Mixed-dataset Training

Innovation

• Direction-Scale Decomposition
• Vision-Language-Action Models
• Action Representation
• Tokenization
• B-spline Tokenizer