Watch Your STEPP: Semantic Traversability Estimation using Pose Projected Features

2025-01-29

AI Summary

Legged robots often exhibit inaccurate traversability assessment on unstructured outdoor terrain. Method: This paper proposes an unsupervised semantic traversability learning approach guided by human walking demonstrations. It leverages DINOv2 to extract pixel-wise self-supervised visual features and employs an encoder-decoder MLP to reconstruct terrain region features. Crucially, reconstruction error is introduced— for the first time—as a confidence metric for traversability, enabling annotation-free anomaly detection and fine-grained traversability estimation. Results: The method is validated on real-world indoor and outdoor scenes using the ANYmal robot. It significantly improves robustness of locomotion decisions across challenging terrains—including loose gravel, steep slopes, and vegetation-covered surfaces. The implementation is publicly available.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Search and Optimization: Learning to Search
• Natural Language Processing: Safety and Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Understanding the traversability of terrain is essential for autonomous robot navigation, particularly in unstructured environments such as natural landscapes. Although traditional methods, such as occupancy mapping, provide a basic framework, they often fail to account for the complex mobility capabilities of some platforms such as legged robots. In this work, we propose a method for estimating terrain traversability by learning from demonstrations of human walking. Our approach leverages dense, pixel-wise feature embeddings generated using the DINOv2 vision Transformer model, which are processed through an encoder-decoder MLP architecture to analyze terrain segments. The averaged feature vectors, extracted from the masked regions of interest, are used to train the model in a reconstruction-based framework. By minimizing reconstruction loss, the network distinguishes between familiar terrain with a low reconstruction error and unfamiliar or hazardous terrain with a higher reconstruction error. This approach facilitates the detection of anomalies, allowing a legged robot to navigate more effectively through challenging terrain. We run real-world experiments on the ANYmal legged robot both indoor and outdoor to prove our proposed method. The code is open-source, while video demonstrations can be found on our website: https://rpl-cs-ucl.github.io/STEPP

Problem

• Autonomous Navigation
• Terrain Traversability
• Legged Robots

Innovation

• DINOv2 Model
• Semantic Traversability Estimation
• ANYmal Robot Navigation