STONE Dataset: A Scalable Multi-Modal Surround-View 3D Traversability Dataset for Off-Road Robot Navigation

2026-03-10

AI Summary

This work addresses the limitations of existing off-road navigation datasets, which lack scalability and multimodal perception capabilities necessary for accurate 3D traversability prediction. To overcome this, we introduce STONE, a large-scale multimodal dataset featuring synchronized surround-view data from a 128-line LiDAR, a six-camera RGB rig, and three 4D imaging radars. We propose a fully automatic, annotation-free method based on Mahalanobis distance to generate geometry-aware, voxel-level 3D traversability ground truth. This approach integrates dense point cloud reconstruction with extraction of terrain geometric attributes—such as slope, elevation, and roughness—enabling efficient, high-quality dataset construction without manual intervention. STONE encompasses diverse environments and day-night conditions, establishing the first voxel-level benchmark for 3D traversability prediction and providing strong unimodal and multimodal baseline models to significantly advance research in off-road robotic navigation.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Data transparency and provenance

Abstract

Reliable off-road navigation requires accurate estimation of traversable regions and robust perception under diverse terrain and sensing conditions. However, existing datasets lack both scalability and multi-modality, which limits progress in 3D traversability prediction. In this work, we introduce STONE, a large-scale multi-modal dataset for off-road navigation. STONE provides (1) trajectory-guided 3D traversability maps generated by a fully automated, annotation-free pipeline, and (2) comprehensive surround-view sensing with synchronized 128-channel LiDAR, six RGB cameras, and three 4D imaging radars. The dataset covers a wide range of environments and conditions, including day and night, grasslands, farmlands, construction sites, and lakes. Our auto-labeling pipeline reconstructs dense terrain surfaces from LiDAR scans, extracts geometric attributes such as slope, elevation, and roughness, and assigns traversability labels beyond the robot's trajectory using a Mahalanobis-distance-based criterion. This design enables scalable, geometry-aware ground-truth construction without manual annotation. Finally, we establish a benchmark for voxel-level 3D traversability prediction and provide strong baselines under both single-modal and multi-modal settings. STONE is available at: https://konyul.github.io/STONE-dataset/

Problem

• 3D traversability prediction
• off-road navigation
• multi-modal dataset
• scalable dataset
• surround-view perception

Innovation

• automated annotation-free labeling
• multi-modal sensor fusion
• 3D traversability prediction
• Mahalanobis-distance-based generalization
• scalable off-road dataset