Mana: Dexterous Manipulation of Articulated Tools

2026-06-11

AI Summary

This work addresses the challenge of coordinating intrinsic degrees of freedom and complex contact interactions in dexterous robotic manipulation of articulated tools. The authors formulate dexterous manipulation as an animation generation problem and propose a coarse-to-fine strategy: starting from procedurally generated grasp keyframes, they combine motion planning with reinforcement learning to automatically synthesize full manipulation trajectories. Drawing inspiration from computer animation—a novel perspective in this domain—the method requires only minimal human annotation to produce diverse tool-use strategies and enables zero-shot sim-to-real transfer. The approach demonstrates successful zero-shot transfer of both grasping and in-hand manipulation across four tools with varying scales and joint configurations, highlighting its generality and scalability.

Abstract

Articulated tool manipulation remains a major challenge in dexterous robotics due to the need to coordinate internal degrees of freedom and contact-rich interactions. While prior work has largely focused on rigid objects, articulated tool use remains underexplored because of its physical complexity and the difficulty of learning functional grasping and manipulation policies. We present Mana (Manipulation Animator), a general sim-to-real framework that reinterprets dexterous manipulation as an animation problem. Inspired by computer animation, Mana employs a coarse-to-fine pipeline that transforms procedurally-generated grasp keyframes into manipulation trajectories through motion planning and reinforcement learning. The data generation process is largely automatic, requiring only a few mouse clicks to specify functional affordances (<1 minute per tool). Across four articulated tools spanning different scales and joint types, Mana achieves zero-shot sim-to-real transfer for both grasping and in-hand manipulation, demonstrating a scalable approach to dexterous articulated tool use.

Problem

• articulated tool manipulation
• dexterous robotics
• functional grasping
• contact-rich interactions
• sim-to-real transfer

Innovation

• dexterous manipulation
• articulated tools
• sim-to-real transfer
• motion planning
• reinforcement learning