🤖 AI Summary
Instance segmentation of unseen objects in cluttered desktop scenes requires both modal (visible) and amodal (full-object) masks, yet acquiring large-scale, diverse, and accurately annotated real-world data remains challenging.
Method: We propose the first end-to-end synthetic data generation framework tailored for amodal segmentation in desktop scenarios. Built upon NVIDIA Isaac Sim Replicator Composer, it implements a Python pipeline that automatically renders photorealistic 3D desktop scenes with varied materials, lighting, and textures, while simultaneously generating rich annotations—including semantic/instance masks, depth maps, occlusion masks, and amodal masks. The framework supports user-defined annotation types and eliminates manual labeling entirely.
Results: Evaluated on the OSD-Amodal dataset, UOAIS-Net trained exclusively on our synthetic data achieves state-of-the-art sim-to-real transfer performance. We publicly release the code, a representative synthetic dataset, and demonstration videos.
📝 Abstract
In this work, we present SynTable, a unified and flexible Python-based dataset generator built using NVIDIA's Isaac Sim Replicator Composer for generating high-quality synthetic datasets for unseen object amodal instance segmentation of cluttered tabletop scenes. Our dataset generation tool can render complex 3D scenes containing object meshes, materials, textures, lighting, and backgrounds. Metadata, such as modal and amodal instance segmentation masks, object amodal RGBA instances, occlusion masks, depth maps, bounding boxes, and material properties can be automatically generated to annotate the scene according to the users' requirements. Our tool eliminates the need for manual labeling in the dataset generation process while ensuring the quality and accuracy of the dataset. In this work, we discuss our design goals, framework architecture, and the performance of our tool. We demonstrate the use of a sample dataset generated using SynTable for training a state-of-the-art model, UOAIS-Net. Our state-of-the-art results show significantly improved performance in Sim-to-Real transfer when evaluated on the OSD-Amodal dataset. We offer this tool as an open-source, easy-to-use, photorealistic dataset generator for advancing research in deep learning and synthetic data generation. The links to our source code, demonstration video, and sample dataset can be found in the supplementary materials.