FRAM: Trajectory-Guided Visual Feature Selection for Compact Language-Conditioned Robot Manipulation

2026-09-25

AI Summary

This study addresses the challenge of deploying large-scale vision-language-action (VLA) models in resource-constrained settings by proposing FRAM. The core innovation lies in pioneering future trajectory prediction as a spatial pointer to guide local visual feature selection, organizing information into positional, state, and motion representations to generate compact policies. Furthermore, supervision labels are automatically derived from camera geometry, eliminating the need for manual annotation. With only 138 million parameters, FRAM achieves a 92.2% success rate on the LIBERO benchmark, approaching the performance of 3.3-billion-parameter models. The method is further validated on a real-world dual-arm UR5e robot, successfully completing cup-stacking tasks. These results demonstrate the feasibility of lightweight yet highly effective deployment for robotic manipulation.

Abstract

Vision-Language-Action models achieve strong performance in robot manipulation, but often require large numbers of parameters. In this work, we propose the Future Representation Action Model (FRAM), a small policy that explicitly links the future end-effector trajectory to the current visual input. FRAM uses the image coordinates of the predicted trajectory as spatial pointers and reads local visual features related to the motion from the current image. This organizes the information for action generation into the reference position (Where), the visual state (What), and the future motion (Future). Trajectory labels are generated automatically from demonstrations and camera geometry, so no manual annotation is needed. With 138.7M parameters, including a frozen language encoder, FRAM reaches an average success rate of 92.2% over the four standard LIBERO suites, close to the 94.2% of $π_0$ with 3.3B parameters. Without extra training, it also reaches an average of 67.3% on LIBERO-Plus. Ablations confirm that both the future trajectory and the local visual features improve performance and robustness. On a real dual-arm UR5e, FRAM stacks cups using only wrist cameras, including choosing and switching between the left and right arms. These results show that selecting visual information based on future motion is an effective way to obtain both high performance and robustness in a small robot policy.

Problem

• Vision-Language-Action models
• robot manipulation
• model compactness
• parameter efficiency

Innovation

• Trajectory-Guided Feature Selection
• Vision-Language-Action Model
• Compact Policy
• Robot Manipulation
• Spatial Pointers