coordinate alignment

This study investigates how to effectively organize heterogeneous robotic demonstration data to enhance cross-embodiment transfer performance. Through controlled simulation experiments, it systematically compares the efficacy of unpaired large-scale data against structured paired data—such as demonstrations aligned by scene, task, or trajectory—under varying morphologies and viewpoints. The findings reveal that, for morphology differences, structured data analogies are more effective than merely increasing data diversity, highlighting distinct data structure requirements for morphology transfer versus viewpoint transfer. By optimizing data composition alone, the approach achieves an average 22.5% improvement in success rate on real-world cross-embodiment transfer tasks, underscoring the critical role of data analogy in enabling effective embodiment-agnostic skill transfer.

This work addresses the challenge of policy transfer across robotic morphologies, which is hindered by embodiment differences that limit the generalization of imitation learning. To overcome this, the authors propose leveraging embodiment-invariant behavioral alignment representations—such as end-effector trajectories, object bounding boxes, and language-based action descriptions—to construct a unified vision-language-action (VLA) model capable of integrating multi-embodiment data and enabling effective cross-embodiment transfer. The approach presents the first systematic evaluation of behavioral alignment representations in sim-to-real transfer, demonstrating substantial performance gains on a newly introduced simulation benchmark. When deployed on real robots, the method improves task success rates by 28% and further enables training augmentation using unlabeled demonstration data lacking action annotations.

Accurate modeling of fine-grained spatial relationships among objects—such as co-occurrence of visually similar objects and multi-level spatial nesting—is critical for real-world robotic task planning in complex indoor environments; however, existing datasets lack 3D positional annotations and explicit spatial relation labels. To address this gap, we introduce SpotSRD, the first spatial-relation-aware dataset designed for real-robot deployment, comprising nearly 1,000 indoor images captured by Boston Dynamics’ Spot robot, with fine-grained annotations of object attributes, 3D coordinates, and 24 spatial relation types. We develop a custom annotation tool and a scene-graph evaluation framework, systematically exposing spatial modeling bottlenecks across six state-of-the-art models. Furthermore, we propose a spatially aware prompting mechanism that injects structured spatial relations into GPT-4o, significantly improving its accuracy on spatial planning tasks. The dataset, annotation tool, and code are publicly released.

Existing tactile simulators struggle to accurately replicate the complex deformations and transduction mechanisms of real sensors, limiting sim-to-real transfer performance. This work proposes a multimodal representation learning framework that maps heterogeneous tactile signals—such as simulated penetration depth and real capacitive readings—into a shared latent space using modality-specific encoders. The model is trained with self-reconstruction, cross-reconstruction, and contrastive alignment losses, enabling zero-shot transfer without requiring high-fidelity simulation of raw sensory signals. By integrating multiphysics simulation to enrich embedding informativeness and leveraging a Warp-accelerated penalty-based contact model for computational efficiency, the approach achieves a 16.7% reduction in force prediction error and a 45.8% decrease in shape reconstruction error. It further demonstrates successful zero-shot cross-modal transfer across multiple downstream tasks and includes an open-sourced, efficient tactile simulation module.

This work addresses two core challenges in large-scale robotic manipulation datasets: (1) designing high-value diversity dimensions to enhance data utility, and (2) efficiently retrieving task-aligned demonstrations from existing datasets. To this end, we introduce a programmable data generation framework that explicitly models controllable diversity variables—including camera pose, object categories, and spatial layout. Our analysis reveals, for the first time, that camera pose and spatial arrangement are critical determinants of both dataset diversity and task alignment. We further propose a task-oriented demonstration retrieval algorithm grounded in geometric-semantic joint alignment. Evaluated on real-world datasets including DROID, our method improves downstream policy performance by up to 70%. Crucially, insights and gains observed in simulation generalize successfully to physical robot platforms, demonstrating robust cross-domain transferability.

Existing end-to-end robotic manipulation approaches rely on 2D visual inputs, which struggle to capture the inherently 3D nature of tasks and suffer from misalignment between perception and action spaces in both spatial and temporal dimensions, limiting generalization. This work proposes a pixel-wise 3D visual representation that constructs aligned vertex maps using camera calibration and depth information, unifying multi-view perception and robot actions within a shared world coordinate frame. To achieve viewpoint-invariant encoding, a bird’s-eye-view (BEV) representation is introduced, complemented by a cross-platform trajectory time-alignment mechanism. The proposed approach substantially mitigates spatiotemporal misalignment between perception and action, significantly enhancing policy generalization and robustness across diverse robots, viewpoints, and human demonstrators. The authors also release pretrained models, code, and a complete data processing pipeline to support reproducibility and further research.

This work addresses the limited cross-platform transferability of existing tactile perception strategies, which are heavily dependent on specific sensor modalities. To overcome this, the study proposes a unified, sensor-agnostic tactile representation by constructing a shared latent space across three heterogeneous tactile modalities—resistive, magnetic, and vision-based. This is achieved through modality-specific encoders, pairwise contact alignment signals, and joint training. The resulting representation enables zero-shot transfer of tactile policies across sensor types. Evaluated on four contact-intensive manipulation tasks, the method significantly improves average success rates from 27.5% to 45.9%, demonstrating effective disentanglement and generalization of cross-modal tactile perception and manipulation.

This work addresses the scarcity of real robot demonstration data and the embodiment gap between human videos and robots in visual appearance and 3D geometry. To bridge this gap, the authors propose the LIDEA framework, which aligns shared 2D visual representations between humans and robots through implicit feature distillation and introduces an embodiment-agnostic explicit 3D geometric alignment strategy to decouple embodiment structure from interaction geometry. By innovatively integrating two-stage transitive distillation, shared latent space modeling, and 3D-aware policy learning, LIDEA is the first method in cross-embodiment imitation learning to simultaneously ensure consistency in 2D representations and invariance in 3D interaction geometry. Experiments demonstrate that the approach achieves baseline-level performance using only 20% of the robot demonstration data and exhibits strong generalization on out-of-distribution tasks.

This work addresses the lack of a systematic organizational framework for embodied intelligence data, which hinders scalability and robot alignment. The authors propose a “Data Pyramid” framework that introduces a hierarchical structure to integrate five heterogeneous data sources—real robot data, UMI-style datasets, first- and third-person videos, simulation data, and general vision-language corpora—organized according to quality, diversity, reusability, and physical fidelity. Through multimodal data evaluation, alignment strategies, and hybrid pretraining, the study systematically analyzes how data composition influences model perception, reasoning, and planning capabilities. The work establishes design principles for data selection and combination in embodied foundation models and identifies six key open challenges, thereby advancing the development of data infrastructure for embodied learning.

Existing vision-language-action (VLA) models primarily focus on semantic alignment, often falling short in capturing the geometric awareness and dynamic manipulability required for embodied tasks. This work proposes GeoAlign, a novel architecture that introduces, for the first time, an ego-state-guided geometric feature querying mechanism. Specifically, the RGB branch is post-trained under RGB-D supervision to generate geometry-enhanced features, which are then dynamically queried using the robot’s ego-state to extract phase-relevant geometric tokens for action prediction. By preserving semantic understanding while achieving spatially precise alignment, GeoAlign substantially improves policy generalization on complex geometric tasks, attaining 99.0% success on LIBERO, an average of 85.3% across three SimplerEnv-Fractal tasks, and 78.8% success rate on eight real-world ALOHA geometric manipulation tasks.