EmbPASS: Towards Cross-Embodiment Open Panoramic Segmentation

2026-10-02

AI Summary

This study addresses the observation shift challenge in cross-embodiment panoramic perception caused by viewpoint and layout discrepancies across platforms. We introduce the novel task of cross-embodiment open-vocabulary panoptic segmentation and construct EmbPASS, a multi-platform benchmark with unified semantic categories. To tackle this task, we propose EPONet, which incorporates a Relation-Aware Metric Adapter (RAMA) to enhance spatial modeling under heterogeneous observations, along with a Content-Adaptive Semantic Transfer (CAST) module to improve cross-platform semantic alignment. Experimental results demonstrate that EPONet achieves state-of-the-art performance on EmbPASS with 35.82% mIoU, surpassing the strongest baseline by 1.10%, while maintaining strong competitiveness on existing benchmarks.

Abstract

Panoramic images provide a complete 360-degree field of view, enabling comprehensive scene understanding for embodied perception. However, heterogeneous embodied platforms exhibit substantial differences in observation viewpoints and spatial layouts, giving rise to cross-embodiment observation shifts that pose additional challenges to consistent and reliable panoramic perception, while systematic studies of this problem remain limited. To bridge this gap, we introduce a new task, termed Cross-Embodiment Open Panoramic Segmentation. Meanwhile, we establish EmbPASS, a multi-platform panoramic semantic segmentation benchmark spanning Vehicle, Drone, Wearable, and Quadruped platforms under a unified semantic taxonomy, providing a testbed for systematically studying cross-embodiment panoramic perception. We further propose EPONet, an open-vocabulary panoramic semantic segmentation network that integrates Relation-Aware Metric Adapter (RAMA) and Content-Adaptive Semantic Transfer (CAST) to enhance spatial modeling and semantic transfer under heterogeneous embodied observations. Extensive experiments show that EPONet achieves the best platform-balanced performance on EmbPASS with 35.82% mIoU, outperforming the strongest baseline by 1.10%, while remaining competitive on existing panoramic segmentation benchmarks. The source code and EmbPASS benchmark will be made publicly available at https://github.com/guopj1/EmbPASS.

Problem

• Cross-Embodiment
• Panoramic Segmentation
• Open-Vocabulary
• Observation Shift
• Embodied Perception

Innovation

• Cross-Embodiment
• Panoramic Segmentation
• Open-Vocabulary
• Relation-Aware Metric Adapter
• Content-Adaptive Semantic Transfer