Fast Trajectory Planner with a Reinforcement Learning-based Controller for Robotic Manipulators

2025-09-22

AI Summary

To address the challenge of real-time, collision-free trajectory planning for robotic manipulators in complex unstructured environments, this paper proposes a fast planning framework that synergistically integrates task-space and joint-space reasoning. Methodologically, we first employ a vision large model (FSA segmentation) to extract scene semantics and combine it with B-spline optimization to generate an initial kinodynamically feasible visual path. Subsequently, we design a proximal policy optimization (PPO) algorithm enhanced with action integration and strategy feedback in joint space, improving obstacle avoidance stability and target convergence accuracy. Our contributions are threefold: (1) the first end-to-end path initialization method integrating vision large model guidance with B-spline-based kinodynamic search; (2) a novel strategy feedback mechanism that enhances PPO’s generalization and robustness in high-dimensional joint space; and (3) support for both sim-to-sim and sim-to-real transfer. Experiments demonstrate significant improvements in planning efficiency, collision-free success rate, and cross-environment adaptability.

Technology Category

• Intelligent Robots: Motion and Path Planning
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Generating obstacle-free trajectories for robotic manipulators in unstructured and cluttered environments remains a significant challenge. Existing motion planning methods often require additional computational effort to generate the final trajectory by solving kinematic or dynamic equations. This paper highlights the strong potential of model-free reinforcement learning methods over model-based approaches for obstacle-free trajectory planning in joint space. We propose a fast trajectory planning system for manipulators that combines vision-based path planning in task space with reinforcement learning-based obstacle avoidance in joint space. We divide the framework into two key components. The first introduces an innovative vision-based trajectory planner in task space, leveraging the large-scale fast segment anything (FSA) model in conjunction with basis spline (B-spline)-optimized kinodynamic path searching. The second component enhances the proximal policy optimization (PPO) algorithm by integrating action ensembles (AE) and policy feedback (PF), which greatly improve precision and stability in goal-reaching and obstacle avoidance within the joint space. These PPO enhancements increase the algorithm's adaptability across diverse robotic tasks, ensuring consistent execution of commands from the first component by the manipulator, while also enhancing both obstacle avoidance efficiency and reaching accuracy. The experimental results demonstrate the effectiveness of PPO enhancements, as well as simulation-to-simulation (Sim-to-Sim) and simulation-to-reality (Sim-to-Real) transfer, in improving model robustness and planner efficiency in complex scenarios. These enhancements allow the robot to perform obstacle avoidance and real-time trajectory planning in obstructed environments. Project page available at: https://sites.google.com/view/ftp4rm/home

Problem

• Generating obstacle-free trajectories for robotic manipulators in cluttered environments
• Reducing computational effort required by traditional motion planning methods
• Improving precision and stability in goal-reaching and obstacle avoidance

Innovation

• Vision-based planner with FSA model and B-spline optimization
• Enhanced PPO algorithm using action ensembles and policy feedback
• Sim-to-Sim and Sim-to-Real transfer for improved robustness