CR-Refiner: An Object-Centric Optimal Transport Reranker for Edit-Conditioned 3D Scene Retrieval

2026-07-21

AI Summary

Existing methods struggle to effectively model the compositional semantics and geometric structural changes of object sets in 3D scenes under natural language editing instructions. To address this challenge, this work proposes CR-Refiner—a training-free reranker that explicitly captures the asymmetric matching between edit instructions and scene objects through a three-stage mechanism: structured query parsing, object-centric unbalanced optimal transport scoring (featuring a 1×G cost matrix and axis-conditioned structural priors), and confidence validation via a frozen large language model (LLM). Evaluated on the newly introduced 3D-CER benchmark, CR-Refiner substantially improves hard-subset R@1 and mAP@10 across all five editing axes for three diverse base retrievers.

Abstract

Edit-conditioned 3D scene retrieval pairs a reference 3D room with a natural-language modification and retrieves rooms from a corpus that satisfy the edit. Three lines of prior work each fall short on this task. 2D composed image retrieval reasons over pixel-level edits and has no primitive for 3D object sets. 3D foundation encoders embed individual objects but cannot compose at the scene level. 3D scene-grounding methods localize references inside a static scene rather than rank modified rooms across a corpus. We present CR-Refiner, a training-free reranker that wraps any base retriever's top-K candidates with three components. A frozen LLM parses the edit into a structured query entity, and each candidate is scored by an unbalanced optimal-transport problem over a 1xG cost matrix coupling category, style, material, and geometry. The unbalanced solver lets the single-entity query drop mass on irrelevant objects, modelling the asymmetry directly. An axis-conditional structural prior adds size-keyword cues for geometric edits and subject-anchor direction cues for spatial edits. An LLM verifier refines the top three candidates with continuous confidence. Because no benchmark evaluates compositional matching over 3D object sets, we additionally release 3D-CER, 4,963 edit-conditioned queries over a 23,381-room indoor corpus across five edit axes, with multi-positive ground truth, CIRR-style hard subsets, and zero-target adversarials. Across three qualitatively distinct base retrievers, CR-Refiner consistently improves hard-subset R@1 and mAP@10 on every edit axis.

Problem

• edit-conditioned retrieval
• 3D scene retrieval
• object-centric representation
• compositional matching
• optimal transport

Innovation

• object-centric retrieval
• optimal transport reranking
• edit-conditioned 3D scene retrieval
• unbalanced optimal transport
• structured query parsing