STAR-VLM: Spatiotemporal Grounding Vision-Language Models for Motion and Velocity Estimation via Automotive Radar Supervision

2026-08-02

AI Summary

This work addresses the challenge that existing vision-language models struggle with metric-level spatiotemporal reasoning—such as motion and velocity estimation—in dynamic scenes, often relying on costly annotations or synthetic data. To overcome this limitation, the study introduces a novel framework that leverages low-cost automotive radar measurements, specifically range and Doppler signals, as unlabeled supervisory cues for training vision-language models. This radar-supervised approach enables accurate understanding of real-world motion dynamics without any manual annotation. By eliminating dependence on synthetic data or complex preprocessing pipelines, the method achieves state-of-the-art performance on both motion classification and metric velocity estimation in driving scenarios, outperforming specialized single-task baselines.

Abstract

Vision-language models (VLMs) are emerging as a key component of embodied intelligence, with growing applications in auto-labeling and end-to-end autonomous driving. However, existing approaches for improving spatiotemporal reasoning in VLMs often rely on complex preprocessing pipelines, expensive human annotations, or synthetic data, which limit scalability and introduce potential sim-to-real gaps. Moreover, although these methods have improved spatiotemporal understanding, they still lack strong metric reasoning capabilities for dynamic scenes, such as estimating object motion in real-world units. Prior work has explored LiDAR-based metric depth supervision to enhance spatial perception, but it does not directly address temporal reasoning. We introduce STAR-VLM, an automotive radar-supervised framework that enhances spatiotemporal VLMs with motion reasoning and metric velocity estimation for autonomous driving. Automotive radar is a low-cost and widely deployed sensor that provides complementary spatiotemporal supervision through range and Doppler measurements. By leveraging these measurements as label-free ground truth during training, STAR-VLM improves the metric spatiotemporal reasoning ability of VLMs. Through experiments on driving scenarios, we show that STAR-VLM achieves state-of-the-art performance on both motion classification and metric velocity estimation, outperforming even task-specific methods designed for each task. These results highlight automotive radar as a scalable and cost-effective source of supervision for building metric-aware spatiotemporal VLMs for real-world autonomous driving.

Problem

• spatiotemporal reasoning
• metric velocity estimation
• motion estimation
• vision-language models
• autonomous driving

Innovation

• automotive radar
• vision-language models
• spatiotemporal reasoning
• metric velocity estimation
• Doppler supervision