🤖 AI Summary
This work proposes a memory- and speed-efficient online semantic SLAM framework to address the challenge of simultaneously achieving geometric consistency, semantic coherence, and computational efficiency in indoor 3D semantic mapping and navigation. By integrating SLAM with semantic mapping, the method constructs a spatiotemporally consistent 3D scene understanding system based on a VGGT tracking head. It processes arbitrarily long video streams via a sliding window, aligns local submaps using camera poses, and lifts 2D instance masks into 3D objects with consistent identity labels. Floor-plane projection is further leveraged to enable efficient navigation support. Evaluated on ScanNet++, the approach achieves competitive point cloud performance while consuming less than 17 GB of GPU memory, enabling real-time interactive assistive navigation with audio feedback.
📝 Abstract
We present SceneVGGT, a spatio-temporal 3D scene understanding framework that combines SLAM with semantic mapping for autonomous and assistive navigation. Built on VGGT, our method scales to long video streams via a sliding-window pipeline. We align local submaps using camera-pose transformations, enabling memory- and speed-efficient mapping while preserving geometric consistency. Semantics are lifted from 2D instance masks to 3D objects using the VGGT tracking head, maintaining temporally coherent identities for change detection. As a proof of concept, object locations are projected onto an estimated floor plane for assistive navigation. The pipeline's GPU memory usage remains under 17 GB, irrespectively of the length of the input sequence and achieves competitive point-cloud performance on the ScanNet++ benchmark. Overall, SceneVGGT ensures robust semantic identification and is fast enough to support interactive assistive navigation with audio feedback.