Learning User Interaction Forces using Vision for a Soft Finger Exosuit

2025-08-04

AI Summary

Modeling distributed soft-tissue contact forces in soft-finger exoskeletons remains challenging due to material nonlinearity and compliance, while conventional embedded sensing suffers from invasiveness and integration complexity. This paper proposes an end-to-end deep learning method that estimates multi-point contact forces non-invasively and in real time using low-resolution grayscale images. A CNN model is trained on diverse deformation-actuation data generated by the SoRoSim simulator, ensuring strong generalization across geometric configurations, actuation states, and visual noise. Crucially, this vision-based force estimation is integrated directly into a closed-loop control architecture for the first time—replacing physical force sensors entirely—and enabling stable, accurate dynamic assistive force regulation. Experimental validation confirms the feasibility and practicality of this vision-driven paradigm for force perception in soft exoskeletons.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Low Level & Physics-based Vision
• Machine Learning: Multimodal Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Wearable assistive devices are increasingly becoming softer. Modelling their interface with human tissue is necessary to capture transmission of dynamic assistance. However, their nonlinear and compliant nature makes both physical modeling and embedded sensing challenging. In this paper, we develop a image-based, learning-based framework to estimate distributed contact forces for a finger-exosuit system. We used the SoRoSim toolbox to generate a diverse dataset of exosuit geometries and actuation scenarios for training. The method accurately estimated interaction forces across multiple contact locations from low-resolution grayscale images, was able to generalize to unseen shapes and actuation levels, and remained robust under visual noise and contrast variations. We integrated the model into a feedback controller, and found that the vision-based estimator functions as a surrogate force sensor for closed-loop control. This approach could be used as a non-intrusive alternative for real-time force estimation for exosuits.

Problem

• Estimating distributed contact forces for soft finger exosuits
• Overcoming nonlinearity and compliance challenges in modeling
• Providing non-intrusive real-time force estimation via vision

Innovation

• Vision-based learning for force estimation
• SoRoSim toolbox for diverse dataset generation
• Non-intrusive real-time feedback control