GOPI: Generation-Oriented 3D Pose Inference for Furniture Insertion from Single-View RGB-D Indoor Scenes

2026-08-07

AI Summary

This work addresses the highly underconstrained problem of inserting new furniture into single-view RGB-D indoor scenes, where the absence of unique scale and positional cues complicates 3D placement. To tackle this challenge, the authors propose a two-stage framework: first, a data-driven iterative inference procedure estimates geometrically plausible 3D poses; second, these poses are projected into pixel-aligned 2D constraints to guide the synthesis of geometrically consistent novel views. By reformulating furniture insertion as a joint optimization of 3D pose inference and image generation, the method introduces a generative-oriented 3D pose reasoning mechanism. Experiments demonstrate that the proposed approach outperforms baseline methods such as direct regression in pose estimation and consistently produces visually coherent results that align closely with underlying 3D geometry across varying furniture scales, confirming its geometric plausibility and generative stability.

Abstract

We study the problem of inserting new furniture into indoor scene images. Under masked single-view 2D image-plane conditioning, however, the physical scale of the inserted furniture relative to the scene cannot be uniquely determined, making physically grounded furniture placement underdetermined from image evidence alone. We therefore reformulate the task as a combination of 3D pose inference and geometry-guided image generation, where estimating a geometrically plausible 3D placement is essential for reliable synthesis. To this end, we propose a two-stage framework. For 3D placement, we introduce GOPI, a generation-oriented 3D pose inference framework that addresses the underdetermined nature of single-view furniture insertion through data-driven iterative inference, producing geometrically plausible object placements. For image generation, we develop a geometry-guided conditioning strategy that projects the inferred 3D pose into the image plane as a pixel-aligned constraint, enforcing consistency between the synthesized image and the underlying 3D geometry. Experimental results validate the proposed framework from both 3D pose estimation and image synthesis perspectives. For 3D placement, GOPI produces poses with stronger geometric feasibility and better consistency with reference layouts than direct regression and vanilla baselines. For image synthesis, our method preserves alignment with the projected 3D geometry across different furniture scales, showing stable projection-generation alignment across the tested furniture scales.

Problem

• furniture insertion
• 3D pose inference
• single-view RGB-D
• geometric consistency
• underdetermined placement

Innovation

• 3D pose inference
• furniture insertion
• geometry-guided generation
• single-view RGB-D
• iterative inference