Real-Time and Accurate Collision-Free Teleoperation via Differentiable Constraint-Based Trajectory Planning

2026-06-07

AI Summary

This work addresses the challenge of self-collisions and environmental collisions in teleoperation, which commonly arise when controlling only the end-effector of a robotic arm. The authors propose a real-time trajectory planning method based on differentiable constraints, introducing—for the first time—a differentiable collision-avoidance constraint derived from convex optimization duality theory into the teleoperation framework. The approach models robot links using capsule primitives and represents environmental obstacles as polyhedra, generating smooth, collision-free trajectories by optimizing the operator’s input. Experiments on both simulation and a physical UR5e platform demonstrate that the method achieves high obstacle modeling fidelity while significantly reducing computational overhead, all within real-time performance bounds, thereby enabling efficient and safe teleoperation.

Abstract

In teleoperation, the human operator typically controls only the end-effector pose, which often leads to self-collisions of the manipulator and collisions with environmental obstacles, since joints and links are not controlled individually. A common strategy to mitigate this issue is to enhance the operator's input using optimal-control-based trajectory planning. As derivative-based solvers require differentiable constraints, existing approaches either approximate robots and obstacles with spheres, reducing geometric accuracy, or approximate derivatives, degrading convergence and increasing computation times. We address these limitations by adapting a recent formulation of differentiable collision-avoidance constraints, based on duality in convex optimization, to the teleoperation setting. The robot is approximated with capsules and the environment with polytopes. We compare the resulting trajectory planning method against state-of-the-art techniques in simulation with varying numbers of obstacles and evaluate it on a UR5e manipulator in a real-world teleoperation test. Results show that our approach achieves lower computation times while enabling more accurate obstacle modeling, leading to smoother and collision-free end-effector teleoperation.

Problem

• teleoperation
• collision avoidance
• trajectory planning
• manipulator
• obstacle avoidance

Innovation

• differentiable collision avoidance
• capsule representation
• convex optimization duality
• real-time teleoperation
• constraint-based trajectory planning