RoboFin3D: A Sim-to-Real Platform for Robotic Surface Finishing

2026-09-29

AI Summary

This study addresses the high cost and poor reproducibility of physical experiments in robotic surface finishing by developing a high-fidelity simulation platform based on Isaac Sim and the Newton engine. Methodologically, it proposes an intermediate-mesh-free signed distance field (SDF) contact computation scheme alongside independent surface-field appearance modeling, and introduces WeldGen to synthesize diverse weld seam assets, thereby enabling low-cost, reproducible grinding and polishing experiments. Experimental results demonstrate that the generated synthetic data significantly enhances visual perception performance: the SAM2 segmentation IoU improves from 77.15% to 84.47%, further reaching 97.41% under sim-to-real mixed training. These findings validate the platform’s high fidelity in both geometric and appearance simulation for industrial robotics applications.

Abstract

Grinding and sanding are fundamental processes in industrial robotic surface finishing. However, physical trials are expensive and consume workpieces, making reproducible experiments difficult. We present RoboFin3D, a sim-to-real platform built on Isaac Sim and the Newton physics engine, that provides physics-based grinding and sanding simulation for cheap and repeatable robotic surface finishing experiments. RoboFin3D utilizes a signed distance field (SDF) to model the changing geometry of the workpiece, enabling contact computation, live updates and rendering without an intermediate mesh. It additionally uses a separate surface field to model progressive surface appearance change during sanding. We also introduce WeldGen, a weld sampling module, to generate weld beads on 8,918 real-world workpiece meshes for providing diverse simulation assets. The simulation parameters are calibrated on real experimental results and our evaluation demonstrates our simulation's fidelity against the real world. We also demonstrate that simulation-generated data can be used to improve the performance of perception models. Simulation-only fine-tuning of SAM2 improves IoU for segmentation of unsanded regions from 77.15% to 84.47%, while combined synthetic and real training reaches 97.41%.

Problem

• Robotic surface finishing
• Sim-to-real
• Grinding and sanding
• Physics simulation
• Reproducible experiments

Innovation

• Sim-to-Real
• Signed Distance Field (SDF)
• Surface Finishing
• WeldGen
• SAM2