Adaptive Contact-Rich Manipulation Through Few-Shot Imitation Learning With Force-Torque Feedback and Pre-Trained Object Representations

2025-01-01

IEEE Robotics and Automation Letters

AI Summary

Controlling contact forces during dense wiping tasks with soft, deformable objects (e.g., sponges) remains challenging due to dynamic variations in surface height and object physical properties, which undermine force control adaptability. Method: This paper proposes a few-shot adaptive force control framework that integrates real-time 6-DoF force–torque closed-loop feedback with CLIP-driven cross-modal object representation transfer, dynamic motion primitive modulation, and few-shot imitation learning. Contribution/Results: The method achieves generalization to unseen surface heights and multiple sponge types from only a small number of demonstrations. Evaluated across 40 cross-domain scenarios, it attains a mean force-tracking accuracy of 96%, outperforming a no-force-feedback baseline by 23×. It demonstrates significantly enhanced robustness across 10 distinct sponge types and 4 surface-height categories, establishing a new state-of-the-art for adaptive manipulation of soft deformable objects in unstructured wiping tasks.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Interaction Techniques and Devices
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Imitation learning offers a pathway for robots to perform repetitive tasks, allowing humans to focus on more engaging and meaningful activities. However, challenges arise from the need for extensive demonstrations and the disparity between training and real-world environments. This paper focuses on contact-rich tasks like wiping with soft and deformable objects, requiring adaptive force control to handle variations in wiping surface height and the sponge's physical properties. To address these challenges, we propose a novel method that integrates real-time force-torque (FT) feedback with pre-trained object representations. This approach allows robots to dynamically adjust to previously unseen changes in surface heights and sponges' physical properties. In real-world experiments, our method achieved 96% accuracy in applying the average reference force, significantly outperforming the previous method that lacked an FT feedback loop, which only achieved 4% accuracy. To evaluate the adaptability of our approach, we conducted experiments under different conditions from the training setup, involving 40 scenarios using 10 sponges with varying physical properties and 4 types of wiping surface heights, demonstrating significant improvements in the robot's adaptability by analyzing force trajectories.

Problem

• Adaptive force control for wiping deformable objects
• Few-shot imitation learning with FT feedback
• Handling surface height and sponge property variations

Innovation

• Integrates Force-Torque feedback for dynamic adjustments
• Uses pre-trained object representations for adaptability
• Achieves high accuracy in force application tasks