PhyVisGen: Physically and Visually High-Fidelity Robotic Manipulation Data Generation

2026-09-22

AI Summary

为解决真实世界数据收集昂贵且难以规模化的问题，PhyVisGen通过高物理和视觉保真度的框架生成大规模机器人操作数据，提高了软体抓取器的数据实用性。

Abstract

Large-scale manipulation demonstrations are essential for learning robust visuomotor policies, yet real-world data collection is expensive and difficult to scale. Simulation offers a promising alternative, but physical and visual discrepancies can limit the transferability of synthetic data, particularly for manipulation with soft grippers. We present PhyVisGen, a physically and visually high-fidelity framework for scalable robotic manipulation data generation. On the physical side, PhyVisGen introduces an arm-gripper coupling method based on the Incremental Potential Contact (IPC), enabling high-fidelity soft contact throughout complete manipulation trajectories. On the visual side, it combines real-scene reconstruction with real-time path tracing to generate visually realistic observations while preserving captured scene appearance. Quantitative evaluations demonstrate the physical and visual fidelity of PhyVisGen. Policies trained exclusively on synthetic manipulation demonstrations achieve 65-95% success across five real-robot tasks, without real-robot demonstration data or policy fine-tuning.

Problem

• Robotic Manipulation
• Data Generation
• Physical Fidelity
• Visual Fidelity
• Simulation

Innovation

• Incremental Potential Contact (IPC)
• high-fidelity soft contact
• real-scene reconstruction
• real-time path tracing