Mars Traversability Prediction: A Multi-modal Self-supervised Approach for Costmap Generation

2025-09-14

AI Summary

This paper addresses the challenge of robust traversability prediction for Mars rovers navigating complex terrains. We propose a robust multimodal self-supervised framework for generating Bird’s-Eye-View (BEV) cost maps. Our method fuses camera and LiDAR data, leveraging IMU-derived ego-motion to automatically generate geometrically consistent traversability labels—eliminating the need for manual annotation. We employ the DINOv3 image encoder and FiLM-based cross-modal fusion, optimizing geometry-driven cost learning via Huber loss and smoothness regularization. Key contributions include: (1) a high-fidelity, reproducible Mars simulation environment; (2) an IMU-driven self-supervised labeling pipeline; and (3) empirical validation of strong robustness against sensor degradation—e.g., under LiDAR sparsification, mean absolute error increases only marginally from 0.0775 to 0.0915. The framework achieves accurate, label-free, and resilient traversability estimation essential for autonomous planetary exploration.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We present a robust multi-modal framework for predicting traversability costmaps for planetary rovers. Our model fuses camera and LiDAR data to produce a bird's-eye-view (BEV) terrain costmap, trained self-supervised using IMU-derived labels. Key updates include a DINOv3-based image encoder, FiLM-based sensor fusion, and an optimization loss combining Huber and smoothness terms. Experimental ablations (removing image color, occluding inputs, adding noise) show only minor changes in MAE/MSE (e.g. MAE increases from ~0.0775 to 0.0915 when LiDAR is sparsified), indicating that geometry dominates the learned cost and the model is highly robust. We attribute the small performance differences to the IMU labeling primarily reflecting terrain geometry rather than semantics and to limited data diversity. Unlike prior work claiming large gains, we emphasize our contributions: (1) a high-fidelity, reproducible simulation environment; (2) a self-supervised IMU-based labeling pipeline; and (3) a strong multi-modal BEV costmap prediction model. We discuss limitations and future work such as domain generalization and dataset expansion.

Problem

• Predicting traversability costmaps for planetary rovers
• Fusing camera and LiDAR data for terrain analysis
• Self-supervised training using IMU-derived labels

Innovation

• Multi-modal fusion of camera and LiDAR data
• Self-supervised training using IMU-derived labels
• Bird's-eye-view costmap prediction with robust geometry