Kinematics-Aware Diffusion Policy with Consistent 3D Observation and Action Space for Whole-Arm Robotic Manipulation

2025-12-19

AI Summary

Existing whole-arm robotic manipulation methods relying solely on end-effector pose often suffer from uncoordinated whole-body motion, poor obstacle avoidance, and limited object interaction capability. To address this, we propose a kinematics-aware diffusion policy: it represents full-arm observations, states, and actions as point sets within a unified 3D space; explicitly incorporates kinematic priors into the diffusion process; and integrates an optimization-driven whole-body inverse kinematics (IK) solver. This formulation eliminates misalignment between joint and task spaces and significantly reduces reliance on demonstration data for nonlinear kinematic modeling. Experiments demonstrate that our method substantially improves task success rates and spatial generalization in both simulation and real-robot settings. It outperforms prior approaches notably in body-aware manipulation tasks—including obstacle-aware grasping and collaborative pushing—where holistic physical reasoning is essential.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Whole-body control of robotic manipulators with awareness of full-arm kinematics is crucial for many manipulation scenarios involving body collision avoidance or body-object interactions, which makes it insufficient to consider only the end-effector poses in policy learning. The typical approach for whole-arm manipulation is to learn actions in the robot's joint space. However, the unalignment between the joint space and actual task space (i.e., 3D space) increases the complexity of policy learning, as generalization in task space requires the policy to intrinsically understand the non-linear arm kinematics, which is difficult to learn from limited demonstrations. To address this issue, this letter proposes a kinematics-aware imitation learning framework with consistent task, observation, and action spaces, all represented in the same 3D space. Specifically, we represent both robot states and actions using a set of 3D points on the arm body, naturally aligned with the 3D point cloud observations. This spatially consistent representation improves the policy's sample efficiency and spatial generalizability while enabling full-body control. Built upon the diffusion policy, we further incorporate kinematics priors into the diffusion processes to guarantee the kinematic feasibility of output actions. The joint angle commands are finally calculated through an optimization-based whole-body inverse kinematics solver for execution. Simulation and real-world experimental results demonstrate higher success rates and stronger spatial generalizability of our approach compared to existing methods in body-aware manipulation policy learning.

Problem

• Addresses whole-arm robotic manipulation with full-arm kinematics awareness.
• Aligns joint space with 3D task space to reduce policy complexity.
• Ensures kinematic feasibility in actions for improved spatial generalizability.

Innovation

• Consistent 3D space representation for states and actions
• Diffusion policy with integrated kinematics priors
• Optimization-based inverse kinematics for whole-body control