NEO: NeRF It Once, Edit It Many Times for Continuous Object Manipulation

2026-07-27

AI Summary

This work addresses the challenge of performing continuous, consistent, and robot-manipulation-friendly object editing in NeRF scenes guided by natural language. To this end, the authors propose NEO, a unified framework that achieves language-specified object removal through neural radiance field resampling and multi-view-consistent progressive inpainting. Additionally, they introduce a knowledge distillation–based approach for direct NeRF editing, employing a teacher–student model to predict the future scene state prior to robotic action execution. The contributions include NEO-Dataset, the first NeRF editing benchmark tailored for robotic manipulation, and state-of-the-art editing performance on object removal and pick-and-place tasks, yielding geometrically consistent, visually coherent results with significantly reduced artifacts.

Abstract

In this paper, we present NEO, a unified framework providing language-guided NeRF editing for robotic manipulation. Our paper introduces (i) a language-guided object removal that combines neural field resampling with multiview-consistent progressive inpainting, (ii) a direct NeRF weight editing method utilizing knowledge distillation, composing original and edited NeRFs via a teacher-student model, enabling coherent modeling of future scene states before a robot executes an action, and (iii) the first benchmark (NEO-Dataset) for quantitatively evaluating NeRF scene editing methods suitable for robot manipulation. We show that our approach outperforms state-of-the-art baselines in scene editing tasks, including object removal and pick-and-place robotic experiments, yielding visually coherent and geometrically consistent edits that reduce artifacts commonly introduced by prior methods.

Problem

• NeRF editing
• language-guided manipulation
• continuous object manipulation
• scene editing
• robotic manipulation

Innovation

• NeRF editing
• language-guided manipulation
• knowledge distillation
• multiview-consistent inpainting
• robotic scene understanding