SemAnCorr: Semantic Anchored Correspondence for Zero-Shot Manipulation Skill Transfer

2026-07-30

AI Summary

This work addresses the challenge of zero-shot manipulation skill transfer across objects with significant geometric disparities but similar functionalities, where inaccurate recovery of local geometric coordinate frames arises from spatially inconsistent feature matching. To overcome this, the authors propose a training-free framework that jointly optimizes object pose and correspondences by identifying semantically consistent anchor regions and propagating constraints across the entire surface via functional maps. This approach establishes dense correspondences that are both semantically accurate and geometrically coherent. By innovatively integrating semantic anchoring with functional mapping, the method achieves a 90.8% semantic accuracy on the PartNet-Mobility benchmark, outperforms existing approaches in geometric consistency, and enables reliable zero-shot skill transfer on real robots using only a single demonstration.

Abstract

Transferring manipulation skills across object instances that share functionality but differ in geometry remains a fundamental challenge in robot learning. While recent correspondence methods leverage dense visual descriptors and 3D feature fields, nearest-neighbor feature matching often produces spatially incoherent correspondences that fail to recover the local geometric frames required for reliable skill transfer. We introduce SemAnCorr, a training-free framework that establishes dense correspondence by selecting semantically consistent anchor regions through joint pose-correspondence optimization and propagating these constraints over the object surface using functional maps. The resulting correspondences preserve both semantic consistency and geometric coherence, enabling object-centric manipulation skills to transfer across geometrically diverse instances. We evaluate SemAnCorr on a dense correspondence benchmark built on PartNet-Mobility, achieving 90.8% semantic accuracy in our benchmark evaluation while improving geometric coherence over recent state-of-the-art baselines. Finally, we show that these improvements translate directly into real-world manipulation performance: using a single demonstration, SemAnCorr enables substantially more reliable zero-shot manipulation skill transfer to previously unseen objects than existing correspondence methods. Videos and additional visualizations are available at [https://semancorr.github.io](https://semancorr.github.io) .

Problem

• zero-shot manipulation
• skill transfer
• dense correspondence
• geometric coherence
• semantic consistency

Innovation

• zero-shot manipulation
• dense correspondence
• semantic consistency
• functional maps
• geometric coherence