Simulation-Driven Imitation Learning for Biosignals-Free Shared-Autonomy Prosthetic Grasping

2026-06-05

AI Summary

This work addresses the challenges of achieving natural, low-effort grasping control for upper-limb prostheses without electromyographic signals and the high cost and poor scalability of collecting real human demonstration data. To overcome these limitations, the authors propose a scalable simulation framework that leverages a virtual wrist-mounted camera to automatically generate diverse, physically plausible human-like grasping demonstrations. By integrating trajectory retargeting with procedurally generated indoor scenes, the framework constructs a large-scale imitation learning dataset to train an end-to-end shared autonomy policy. The resulting system achieves, for the first time, fully bio-signal-free prosthetic grasping control, attaining over 90% success rates across three real-world scenarios. It significantly outperforms existing approaches and demonstrates exceptional generalization across objects and environments, along with strong potential for efficient real-world deployment.

Abstract

Biosignals-free shared-autonomy control of upper-limb prosthetic hands aims to enable natural and low-effort manipulation without relying on EMG or other physiological signals. Recent imitation-learning-based approaches have shown promising results, but their scalability is limited by the cost and variability of collecting large amounts of real-world human demonstration data. In this work, we present a scalable simulation framework that automatically generates diverse reach-to-grasp demonstrations from a wrist-mounted virtual camera. The framework combines physically feasible grasp synthesis, natural reaching trajectories retargeting, and reach--grasp--lift execution in procedurally generated indoor environments. It records wrist-view observations, proprioception, and actions to build a large-scale demonstration dataset for imitation learning. Through extensive simulation benchmarks, we evaluate object and scene generalization and compare several representative state-of-the-art imitation learning methods. Results show that the simulated demonstrations are sufficiently rich and consistent for effective policy learning. In three realistic settings, the learned sim-to-real policy achieves over 90\% grasp success, surpasses baseline methods, and exhibits stronger generalization, highlighting the promise of simulation-driven training for biosignals-free shared-autonomy prosthetic grasping. The demonstrations are available at \href{https://sites.google.com/view/sim-prosthetic-grasp/home}{https://sites.google.com/view/sim-prosthetic-grasp/home}.

Problem

• biosignals-free
• shared-autonomy
• prosthetic grasping
• imitation learning
• simulation

Innovation

• simulation-driven imitation learning
• biosignals-free control
• shared-autonomy prosthetics
• procedural environment generation
• sim-to-real transfer