OREO: Fidelity Alignment in 3D Generation via On-the-fly Rendering-Editing Optimization

2026-09-24

AI Summary

This study addresses the limited visual fidelity of 3D generative models by proposing OREO, a framework that discards reliance on static datasets in favor of a dynamic optimization loop. Specifically, OREO leverages 2D diffusion priors to guide 3D generation and introduces a novel reinforcement editing mechanism. This mechanism rectifies rendered views into pseudo-targets for self-supervised learning, substantially improving view quality while strictly preserving geometric and content consistency. Experimental results demonstrate that the proposed approach effectively enhances pretrained baselines, yielding 3D assets with significantly improved visual realism.

Abstract

Despite recent advancements in 3D generation, models often struggle to produce assets with high visual fidelity. To bridge this gap, we propose OREO, an alignment framework that enhances the realism of 3D generators by leveraging rich 2D diffusion priors. Instead of relying on static datasets, OREO establishes a dynamic optimization loop that produces on-the-fly edited renderings as 2D pseudo-targets. At its core, we introduce Reinforced Editing, which utilizes a 2D model to refine rendered views of the 3D output, enhancing their overall visual fidelity while preserving the underlying geometry, viewpoint, and content. These refined views serve as high-quality supervision targets, enabling the 3D generator to learn from its own generated samples and progressively improve its visual quality. Experiments demonstrate that OREO effectively improves upon pre-trained baselines, producing 3D assets with enhanced visual realism.

Problem

• 3D generation
• visual fidelity
• 3D assets
• realism

Innovation

• 3D Generation
• Fidelity Alignment
• Reinforced Editing
• Diffusion Priors
• Dynamic Optimization