WARL: Wrench-Augmented Reinforcement Learning for Task-Agnostic Learning in Legged Robots

2026-07-27

AI Summary

This work addresses the limited exploration capability in joint action spaces for legged robot reinforcement learning by introducing wrench (force/torque) commands into the action space and designing a success-rate-based switching curriculum that enhances early-stage exploration efficiency. The proposed approach achieves efficient cross-terrain, multi-task learning without requiring task-specific rewards or intricate curriculum engineering. Experimental validation on a quadrupedal robot demonstrates the effectiveness of the wrench-augmented policy and the curriculum mechanism: the final policy operates solely with joint-level control yet exhibits strong generalization and robustness. Ablation studies further confirm that the gradual removal of wrench commands through the curriculum is crucial for performance improvement.

Abstract

While reinforcement learning for legged robots has achieved high motor performance, it has been constrained by the limited exploration capability of actions confined to the joint space. To address this issue, this study proposes a new method, Wrench-Augmented Reinforcement Learning (WARL), which introduces a wrenche (force and torque) into the action space. The proposed method combines wrench-guided exploration with a success rate-based curriculum mechanism to expand exploration capabilities in the early stages of learning, with the ultimate goal of acquiring behaviors based solely on joint control. Experiments using a quadruped robot demonstrated that WARL can learn robustly across diverse terrains and motor tasks without requiring terrain-specific reward adjustments or complex curriculum designs. Furthermore, an ablation study verified the effectiveness of the Switching Curriculum, which gradually eliminates the wrench. On the other hand, we also show that introducing a wrench can encourage behaviors that do not sufficiently exploit the robot's physical embodiment. These findings suggest that while wrench-based exploration enhancement is effective for improving learning efficiency, designing it in a way that is consistent with the robot's physical structure is a critical future challenge.

Problem

• reinforcement learning
• legged robots
• action space
• exploration capability
• wrench

Innovation

• Wrench-Augmented Reinforcement Learning
• task-agnostic learning
• legged robots
• curriculum learning
• action space augmentation