CT-VAM: A Cerebello-Thalamic-Inspired Vision-Action Model for Efficient Visuomotor Control

2026-06-08

AI Summary

This work addresses the efficiency bottleneck in high-frequency, low-level robot control caused by repeatedly parsing language instructions. Inspired by cerebellar-thalamic neural mechanisms, the authors propose CT-VAM, a lightweight vision-action model that decouples task intent from real-time motor execution. CT-VAM employs a Thalamic Action Routing Stream (TARS) architecture to separate task, visual, and motor information flows, integrating dual-view vision, proprioception, optical-flow-consistent image inpainting, and a conditional attention decoder. With only 68 million parameters, the model achieves success rates on the LIBERO benchmark comparable to significantly larger vision-language-action models while substantially reducing inference latency, enabling efficient and robust deployment on resource-constrained physical robotic platforms.

Abstract

Vision-language-action models have shown strong promise for robot manipulation, yet raw language is primarily needed to specify task intent rather than to be repeatedly processed during high-frequency low-level execution. Motivated by this separation, we propose a cerebello-thalamic-inspired vision-action model (CT-VAM) for efficient task-conditioned visuomotor control. CT-VAM acts as a compact local execution policy that predicts action chunks from dualview visual observations, proprioception, and a lightweight task condition, potentially enabling a practical cloud-edge paradigm in which high-level semantic reasoning can be handled by large models while fast closed-loop control runs on local hardware. To fuse heterogeneous inputs effectively, CT-VAM introduces TARS (Thalamic Action Routing Stream), a stream-separated conditional attention decoder that independently routes action, visual and task streams, preventing dense sensory tokens from overwhelming compact task-relevant conditions. With only 68M parameters, CT-VAM achieves LIBERO success rates competitive with substantially larger VLA models, while reducing inference latency. Together with flow-consistent inpainting for asynchronous chunk execution, CT-VAM supports high-frequency control and demonstrates robust realworld deployment on resource-constrained robotic platforms.

Problem

• visuomotor control
• task-conditioned execution
• low-latency inference
• resource-constrained robotics
• vision-action modeling

Innovation

• CT-VAM
• TARS
• visuomotor control
• stream-separated attention
• cloud-edge robotics