Closed-Loop Sim-to-Real Reinforcement Learning for Deformable Microfiber Shape Control

2026-05-20

AI Summary

This work addresses the challenge of modeling microscale surface and interfacial interactions in micromanipulation, which severely limits the sim-to-real transfer performance of conventional model-based approaches. The authors propose a closed-loop reinforcement learning framework that trains geometric manipulation policies in a frictionless, simplified simulator and iteratively corrects unmodeled contact effects in the physical system using real-time visual feedback. For the first time, this approach enables direct deployment of policies trained purely in simulation to a dual-gripper micromanipulation system without fine-tuning or domain adaptation, successfully controlling the deformation of microfibers. The method achieves an average point error of 270 ± 80 μm across 24 initial configurations and demonstrates sub-millimeter shape control accuracy on nine samples spanning three diameters and lengths, confirming its robustness and generalization capability.

Abstract

Autonomous contact-based micromanipulation is challenging because surface and interfacial interactions at the microscale are difficult to model accurately, limiting the use of conventional model-based control and sim-to-real learning. We present a closed-loop sim-to-real reinforcement learning (RL) approach for microfiber shape control on a surface. The central idea is to train geometric shape regulation in a simplified frictionless simulator and rely on real-time visual feedback during deployment to iteratively correct the observed effects of unmodeled surface interactions. An RL policy trained entirely in simulation is transferred directly to a physical dual-gripper micromanipulation system operating at 40 Hz, without retraining or domain adaptation. Using silk microfibers as a testbed, the policy achieves a mean point-wise shape error of 270 $\pm$ 80 $μ$m across twenty-four diverse initial configurations. Across nine specimens covering all combinations of three fiber diameters (50, 80, and 120 $μ$m) and three manipulated lengths (10 mm, 15mm, and 20 mm), the same policy achieves sub-millimeter final shape error without any retraining or retuning. These results show that a policy learned in a simplified simulator can achieve repeatable real-world microfiber shape regulation under surface contact, provided that the task-relevant effects of the sim-to-real mismatch remain observable and correctable within the closed feedback loop.

Problem

• micromanipulation
• deformable microfiber
• sim-to-real
• surface interaction
• shape control

Innovation

• closed-loop sim-to-real
• reinforcement learning
• micromanipulation
• deformable microfiber control
• visual feedback