Robust Model-Based In-Hand Manipulation with Integrated Real-Time Motion-Contact Planning and Tracking

2025-05-08

AI Summary

To address the inefficiency of online planning and poor robustness to modeling errors arising from frequent dynamic contact transitions during in-hand manipulation with dexterous hands, this paper proposes a hierarchical motion-contact coordination framework. At the high level, contact-implicit model predictive control (CIMPC) enables real-time joint motion and contact planning; at the low level, a hand-specific force-motion coupled dynamic model is integrated with tactile-feedback-driven closed-loop servo tracking, establishing a unified “planning-and-tracking” architecture. This design supports online compensation for modeling errors and rapid recovery from external disturbances. Evaluated on a physical robot platform across five highly contact-intensive manipulation tasks, the framework achieves superior accuracy, robustness, and real-time performance—demonstrating an average planning latency of under 20 ms—outperforming existing model-based approaches.

Technology Category

• Intelligent Robots: Motion and Path Planning
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Robotic dexterous in-hand manipulation, where multiple fingers dynamically make and break contact, represents a step toward human-like dexterity in real-world robotic applications. Unlike learning-based approaches that rely on large-scale training or extensive data collection for each specific task, model-based methods offer an efficient alternative. Their online computing nature allows for ready application to new tasks without extensive retraining. However, due to the complexity of physical contacts, existing model-based methods encounter challenges in efficient online planning and handling modeling errors, which limit their practical applications. To advance the effectiveness and robustness of model-based contact-rich in-hand manipulation, this paper proposes a novel integrated framework that mitigates these limitations. The integration involves two key aspects: 1) integrated real-time planning and tracking achieved by a hierarchical structure; and 2) joint optimization of motions and contacts achieved by integrated motion-contact modeling. Specifically, at the high level, finger motion and contact force references are jointly generated using contact-implicit model predictive control. The high-level module facilitates real-time planning and disturbance recovery. At the low level, these integrated references are concurrently tracked using a hand force-motion model and actual tactile feedback. The low-level module compensates for modeling errors and enhances the robustness of manipulation. Extensive experiments demonstrate that our approach outperforms existing model-based methods in terms of accuracy, robustness, and real-time performance. Our method successfully completes five challenging tasks in real-world environments, even under appreciable external disturbances.

Problem

• Enables robust in-hand manipulation with real-time planning and tracking
• Addresses modeling errors and contact complexity in dexterous robotics
• Integrates motion-contact optimization for improved accuracy and disturbance recovery

Innovation

• Hierarchical real-time motion-contact planning and tracking
• Contact-implicit model predictive control for references
• Force-motion model with tactile feedback for robustness