NeRFifyMesh: Optimizing Neural Radiance Fields from Textured Meshes for Robotics Scene Building

2026-10-07

AI Summary

This study addresses the bottleneck of inefficiently converting 3D mesh assets into Neural Radiance Fields (NeRFs) for robotic simulation by proposing a direct conversion pipeline from textured meshes to point-based radiance fields. By sampling mesh geometry and textures to construct ground-truth representations, this method generates neural radiance fields without requiring camera pose sampling or multi-view rendering, thereby substantially simplifying the training process. Experimental results demonstrate that the proposed approach achieves rendering quality comparable to existing baselines while enabling the construction of unified NeRF scenes. These scenes support accurate geometry extraction and collision simulation, effectively accelerating the development of robotic algorithms.

Abstract

In robotics, scene representation plays a pivotal role in understanding and interacting with the environment. The advent of Neural Radiance Fields (NeRF) and its variants, as a novel representation, has opened a new frontier of research. In applications such as semantic mapping and simulation, roboticists aim to build scenes using multiple NeRF models, each representing an object. While extensive datasets of 3D mesh models already exist, there is an urgent need to develop tools to convert these assets to NeRF models for rapid algorithm development and testing. This paper presents a new pipeline for converting existing mesh models to NeRF representations by artificially generating a ground truth point-based radiance field through sampling mesh geometry and texture. This approach alleviates the need for camera-based sampling or rendering multi-view images of the original mesh to train the NeRF model. Extensive benchmarking demonstrates that our method yields comparable rendering quality to the baselines. Additionally, the application of this representation is shown by constructing unified NeRF scenes and performing collision simulations with extracted geometry.

Problem

• Neural Radiance Fields
• 3D mesh
• robotics
• scene representation
• asset conversion

Innovation

• Neural Radiance Fields
• Textured Meshes
• Point-based Radiance Field
• Robotics Scene Building
• Mesh-to-NeRF Conversion