🤖 AI Summary
To address the high cognitive load imposed on users of multifunctional prosthetic hands due to manual prehensile posture selection, this paper proposes an eye-in-hand shared-control framework. It leverages RGB temporal streams from a wrist-mounted camera to perform real-time object-part detection and optimal grasp-type classification. Key contributions include: (1) a fine-grained part-aware prehensile classification mechanism enabling region-specific grasp adaptation for the same object; (2) a physics-engine-based hand–object interaction synthesis and rendering pipeline to mitigate scarcity of real-world annotated data; and (3) a sensorized human grasping ground-truth acquisition platform, with embedded, low-latency deployment of the model on the Hannes prosthetic hand. Experiments demonstrate that models trained on synthetic data achieve superior generalization compared to those trained on real data. The source code and dataset are publicly released.
📝 Abstract
We consider the task of object grasping with a prosthetic hand capable of multiple grasp types. In this setting, communicating the intended grasp type often requires a high user cognitive load which can be reduced adopting shared autonomy frameworks. Among these, so-called eye-in-hand systems automatically control the hand pre-shaping before the grasp, based on visual input coming from a camera on the wrist. In this paper, we present an eye-in-hand learning-based approach for hand pre-shape classification from RGB sequences. Differently from previous work, we design the system to support the possibility to grasp each considered object part with a different grasp type. In order to overcome the lack of data of this kind and reduce the need for tedious data collection sessions for training the system, we devise a pipeline for rendering synthetic visual sequences of hand trajectories. We develop a sensorized setup to acquire real human grasping sequences for benchmarking and show that, compared on practical use cases, models trained with our synthetic dataset achieve better generalization performance than models trained on real data. We finally integrate our model on the Hannes prosthetic hand and show its practical effectiveness. We make publicly available the code and dataset to reproduce the presented results11https://github.com/hsp-iit/prosthetic-grasping-simulation.
