Grasp, Handover, Rotate: Bimanual Object Reorientation via Compositional Diffusion and Energy-Based Optimization

2026-07-23

AI Summary

This work addresses the challenge of object placement for dual-arm robots in complex scenarios involving collisions, kinematic constraints, or suboptimal end-effector poses, where direct placement is often infeasible. To tackle this, the authors propose BiCompoDiff, a novel framework that, for the first time, integrates a pretrained grasping diffusion model with an energy-based dual-arm motion planner. By leveraging differentiable inverse kinematics, gradient-guided reverse diffusion, and annealed MCMC sampling, the method jointly optimizes grasp selection, inter-hand transfer, regrasping, and motion planning to generate coordinated strategies under multiple constraints. Experiments demonstrate that the approach improves task success rates by over 20% and trajectory smoothness by 37% in simulation, while real-robot evaluations confirm its strong sim-to-real transferability and robustness.

Abstract

Bimanual object reorientation - picking an object, handing it over between two arms, and placing it in a desired target pose - is valuable when direct placement from the initial grasp is infeasible due to collisions, kinematic constraints, or poor final orientation. However, achieving this under multiple competing objectives remains challenging. We introduce BiCompoDiff, a compositional diffusion and energy-based framework that jointly optimizes grasp selection, handover, regrasp, and motion planning under multiple constraints. By combining a pretrained grasp diffusion model with bimanual planning energy-based models (EBMs), our method injects gradient guidance during reverse diffusion to enforce collision avoidance, trajectory smoothness (via differentiable inverse kinematics), handover feasibility, and regrasp safety. Annealed MCMC sampling further refines grasp poses over the composite energy landscape. Experiments across diverse simulated household reorientation tasks demonstrate that BiCompoDiff achieves over 20% higher success rates and up to 37% smoother trajectories (measured by joint displacement) compared to strong sampling-based baselines. Real-world validation confirms effective sim-to-real transfer and robust performance on challenging scenes.

Problem

• bimanual object reorientation
• grasp
• handover
• motion planning
• collision avoidance

Innovation

• compositional diffusion
• energy-based models
• bimanual manipulation
• grasp optimization
• sim-to-real transfer