Robotic In-Hand Manipulation for Large-Range Precise Object Movement: The RGMC Champion Solution

2025-02-11

AI Summary

This work addresses the challenge of large-range, high-precision in-hand object pose regulation for dexterous robotic hands. We propose a model-free, pretraining-free real-time kinematic trajectory optimization method that achieves simultaneous centimeter-scale (>5 cm) repositioning and sub-millimeter positioning accuracy using only finger joint actuation—without requiring object geometry models or deep learning components—while maintaining persistent, stable grasping. The core contribution is an online optimization framework that tightly integrates contact stability constraints with finger kinematic planning, significantly enhancing generalization across unseen objects and deployment efficiency. Evaluated on the ICRA 2024 RGMC In-Hand Manipulation Challenge, our method secured first place, demonstrating robust adaptability to unknown real-world objects under unstructured conditions.

Technology Category

• Intelligent Robots: Manipulation
• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: Motion & Tracking

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

In-hand manipulation using multiple dexterous fingers is a critical robotic skill that can reduce the reliance on large arm motions, thereby saving space and energy. This letter focuses on in-grasp object movement, which refers to manipulating an object to a desired pose through only finger motions within a stable grasp. The key challenge lies in simultaneously achieving high precision and large-range movements while maintaining a constant stable grasp. To address this problem, we propose a simple and practical approach based on kinematic trajectory optimization with no need for pretraining or object geometries, which can be easily applied to novel objects in real-world scenarios. Adopting this approach, we won the championship for the in-hand manipulation track at the 9th Robotic Grasping and Manipulation Competition (RGMC) held at ICRA 2024. Implementation details, discussion, and further quantitative experimental results are presented in this letter, which aims to comprehensively evaluate our approach and share our key takeaways from the competition. Supplementary materials including video and code are available at https://rgmc-xl-team.github.io/ingrasp_manipulation .

Problem

• In-hand manipulation for precise object movement
• Achieving high precision and large-range movements
• Maintaining constant stable grasp during manipulation

Innovation

• Kinematic trajectory optimization
• No pretraining required
• Suitable for novel objects