DexMani: Human-Derived Manipulability Guidance for Dexterous Rotation

2026-08-01

AI Summary

Existing reinforcement learning approaches struggle to model the influence of contact transitions on dexterous in-hand object rotation. This work proposes a novel framework that, for the first time, explicitly incorporates the evolution of manipulability under contact conditions—extracted from human demonstrations—as a guiding signal into reinforcement learning to directly shape policy learning. The method requires no hand-specific tuning and enables cross-embodiment transfer of dexterous rotation skills. Experiments on the Shadow Hand, Allegro Hand, XHand, and LEAP Hand demonstrate that the approach significantly improves task success rates—reaching 57.5% on the LEAP Hand—and produces smoother manipulation trajectories, thereby validating its generality and effectiveness.

Abstract

Dexterous object rotation is a sequential contact problem: each support, release, and re-contact decision must both produce the desired object motion, and prepare the hand configuration for continued rotation. Existing reinforcement learning methods discover such movement patterns through trial and error on specific robotic hand embodiments, without explicitly accounting for how each contact transition affects the hand's ability to sustain object rotation in subsequent steps. We introduce DexMani, a framework that transfers human demonstrations as contact-conditioned manipulability evolution. This prior captures how successful human contact transitions reshape the object-rotation directions available to the hand. DexMani then learns this manipulability evolution and uses it to guide downstream reinforcement learning, enabling rotation skills to be acquired across robot embodiments with distinct kinematics and active-contact configurations. Across the Shadow Hand, Allegro Hand, and XHand, DexMani achieves the highest success rates in every evaluated setting for both seen and unseen objects. DexMani reaches an average success rate of 57.5% on LEAP Hand, outperforming other baselines and producing smoother rotatory motions. Project site: https://dexmani.github.io

Problem

• dexterous manipulation
• object rotation
• contact transition
• manipulability
• reinforcement learning

Innovation

• dexterous manipulation
• manipulability guidance
• human demonstration transfer
• contact transition
• reinforcement learning