Learning Dexterous Bimanual Catch Skills through Adversarial-Cooperative Heterogeneous-Agent Reinforcement Learning

2025-02-17

AI Summary

Bimanual robotic grasping suffers from poor hand coordination and limited generalization across diverse objects. Method: We propose an adversarial-cooperative heterogeneous-agent reinforcement learning framework. It comprises two specialized agents: a “throwing” agent that dynamically increases task difficulty, and a “catching” agent that learns adaptive bimanual coordination policies. We introduce a novel adversarial reward mechanism that drives progressive task difficulty escalation, and train the framework in PyBullet/Mujoco simulation environments for multi-object generalization. Contribution/Results: This work is the first to integrate heterogeneous agents and adversarial rewards into bimanual throwing-and-catching tasks, significantly improving control flexibility and robustness. Our method achieves stable grasping across 15 object classes with varying shapes and sizes. The average catching reward improves by approximately 100% over single-agent baselines, demonstrating substantial advances in both task complexity handling and cross-object generalization capability.

Technology Category

• Multiagent Systems: Adversarial Agents
• Intelligent Robots: Manipulation
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Robotic catching has traditionally focused on single-handed systems, which are limited in their ability to handle larger or more complex objects. In contrast, bimanual catching offers significant potential for improved dexterity and object handling but introduces new challenges in coordination and control. In this paper, we propose a novel framework for learning dexterous bimanual catching skills using Heterogeneous-Agent Reinforcement Learning (HARL). Our approach introduces an adversarial reward scheme, where a throw agent increases the difficulty of throws-adjusting speed-while a catch agent learns to coordinate both hands to catch objects under these evolving conditions. We evaluate the framework in simulated environments using 15 different objects, demonstrating robustness and versatility in handling diverse objects. Our method achieved approximately a 2x increase in catching reward compared to single-agent baselines across 15 diverse objects.

Problem

• Bimanual robotic catching
• Coordination and control challenges
• Adversarial-cooperative learning framework

Innovation

• Heterogeneous-Agent Reinforcement Learning
• Adversarial reward scheme
• Bimanual coordination control