Stability-aware Residual Reinforcement Learning Framework for Robotic Manipulator Disturbance Compensation

2026-09-18

AI Summary

研究提出了一种结合分析观测器与强化学习策略的残差强化学习扰动观测器框架，以解决机器人操作器中的参数不确定性、非线性摩擦及复合扰动问题。

Abstract

Although conventional controllers and disturbance observers (DOBs) are the standard for precision tracking in manipulators, they suffer from parameter uncertainty, nonlinear friction, and compound disturbances. This study proposes a residual reinforcement learning DOB framework that pairs an analytical observer with an RL policy. The deterministic baseline operates within a reliable region, whereas the RL policy explicitly targets the residuals that the model cannot capture. To make this compensation disturbance-aware, an estimator network aligns the observation history with a privileged disturbance context, organizing the latent space by disturbance regime and enabling rapid adaptation across disturbance transitions. To guarantee stability, we derived and enforced a state-dependent action bound on the RL policy from an input-to-state stability (ISS) analysis such that the closed loop provably confines the tracking error to a certified envelope for arbitrary policy outputs. Experiments on a 6-DOF manipulator demonstrated consistent improvements in disturbance estimation and tracking, including a 27.8% tracking-error reduction on real hardware under zero-shot sim-to-real transfer and a 38.0% reduction under a base-vibration disturbance that was not observed during training.

Problem

• controller
• disturbance observer
• parameter uncertainty
• nonlinear friction
• compound disturbances

Innovation

• residual reinforcement learning
• disturbance observer
• input-to-state stability
• adaptive compensation
• zero-shot sim-to-real transfer