3DGS-CD: 3D Gaussian Splatting-based Change Detection for Physical Object Rearrangement

2024-11-06

arXiv.org

AI Summary

This work addresses 3D change detection for physical object rearrangement in unstructured scenes. We propose the first end-to-end framework based on 3D Gaussian Splatting (3DGS), requiring only sparse, unaligned RGB images (as few as one) without depth input, category priors, or object models. Our method integrates EfficientSAM for zero-shot 2D segmentation with a cross-view geometric consistency–driven 3D change fusion mechanism, enabling unsupervised, category-agnostic localization and reconstruction of rearranged objects. Key contributions include: (i) the first paradigm for 2D→3D change mask estimation; and (ii) a lightweight, online-updatable 3DGS-based change-aware model. Evaluated on real-world datasets, our approach achieves a 14% accuracy improvement over state-of-the-art radiance field methods while accelerating inference by three orders of magnitude (18 seconds per instance), supporting robotic workspace reset and dynamic 3D scene maintenance.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

We present 3DGS-CD, the first 3D Gaussian Splatting (3DGS)-based method for detecting physical object rearrangements in 3D scenes. Our approach estimates 3D object-level changes by comparing two sets of unaligned images taken at different times. Leveraging 3DGS's novel view rendering and EfficientSAM's zero-shot segmentation capabilities, we detect 2D object-level changes, which are then associated and fused across views to estimate 3D change masks and object transformations. Our method can accurately identify changes in cluttered environments using sparse (as few as one) post-change images within as little as 18s. It does not rely on depth input, user instructions, pre-defined object classes, or object models -- An object is recognized simply if it has been re-arranged. Our approach is evaluated on both public and self-collected real-world datasets, achieving up to 14% higher accuracy and three orders of magnitude faster performance compared to the state-of-the-art radiance-field-based change detection method. This significant performance boost enables a broad range of downstream applications, where we highlight three key use cases: object reconstruction, robot workspace reset, and 3DGS model update. Our code and data will be made available at https://github.com/520xyxyzq/3DGS-CD.

Problem

• 3D Object Detection
• Position Change Recognition
• Complex Environment

Innovation

• 3DGS-CD
• Object Position Change Detection
• High Accuracy and Speed