implicit neural fields

Existing feedforward 3D scene understanding methods suffer from coarse semantic representations, low-fidelity geometric reconstruction, significant semantic noise, and reliance on dense view inputs—leading to high deployment costs. This paper proposes an end-to-end, sparse-view-driven framework for holistic 3D scene understanding, unifying geometric, appearance, and language-level semantic modeling. Key contributions include: (1) a semantic-aware anisotropic Gaussian field representation; (2) multi-source semantic feature fusion via a cross-view cost volume; and (3) a two-stage implicit semantic field distillation mechanism enabling open-vocabulary, promptable 3D segmentation. Experiments demonstrate substantial improvements over baselines such as LSM on both promptable and open-vocabulary 3D segmentation benchmarks. Our method achieves finer-grained geometry reconstruction, markedly reduces semantic noise, and supports real-time augmented reality interaction.

Neural network representations suffer from opacity, knowledge overwrite, and structural uninterpretability—hindering continual learning in visual domains. Method: This paper introduces the first structured continual learning framework tailored for visual space. It integrates environmental dynamics modeling, network refinement, and rerelation mechanisms to explicitly and hierarchically model object core structures and critical sub-variants in an incremental manner, thereby avoiding information diffusion and catastrophic forgetting inherent in conventional iterative optimization. Contribution/Results: We achieve the first instance of structured continual learning for visual representations, yielding compact, interpretable, and generalizable hierarchical representations. Experiments on MNIST demonstrate zero-forgetting 2D shape detection, significant reduction in representation size, consistently improved accuracy, and emergent cross-task generalization—marking a paradigm shift beyond black-box neural representations.

Current 3D semantic scene graph prediction methods rely on graph neural networks but suffer from insufficient discriminability and representational capacity in object and relational feature encoding. To address this, we propose a decoupled representation learning framework: first, a highly discriminative object feature encoder is designed, integrating geometric-semantic multimodal fusion; second, an object-centric contrastive pre-training strategy is introduced to explicitly decouple object representation learning from graph structure prediction. Crucially, our method requires no architectural modifications to downstream graph inference modules and can be seamlessly integrated as a plug-in enhancement. Evaluated on the 3DSSG benchmark, our approach significantly outperforms state-of-the-art methods, achieving consistent improvements in both object classification and relationship prediction—the two core evaluation metrics—thereby validating the effectiveness of decoupled representation learning for 3D scene semantic understanding.

This study investigates how object-centric (OC) representations enhance compositional generalization and structured reasoning in visual question answering (VQA), and analyzes their complementarity with large vision-language foundation models (e.g., ViT, CLIP). We introduce the first large-scale empirical framework, evaluating over 600 downstream VQA models across 15 upstream representation types—including OC models (Slot Attention, IODINE)—and incorporating multi-stage fine-tuning and prompting strategies. Our key contributions are: (1) the first empirical validation that OC representations substantially improve compositional generalization on both synthetic (CLEVR) and real-world (GQA) benchmarks; (2) a hybrid paradigm integrating OC representations with foundation models; and (3) experimental results demonstrating an average accuracy gain of 3.2% and a 21% improvement in robustness, revealing a synergistic division of labor—OC representations excel at structured, part-based reasoning, while foundation models support open-domain semantic understanding.

This work addresses the challenges of natural language–driven semantic understanding in dynamic 4D scenes, where weak contextual reasoning, view-dependent noise, and cross-spatiotemporal semantic inconsistency hinder performance. The authors propose a query-time reasoning framework that, for the first time, integrates a multi-view semantic consensus mechanism with 4D Gaussian splatting reconstruction and neural field optimization to achieve structured 4D semantic grounding while preserving geometric consistency. By fusing multi-view, multi-frame 2D semantic predictions, the method effectively supports complex linguistic queries involving object attributes, actions, spatial relations, and multi-object interactions. Evaluated on the newly introduced Panoptic-L4D benchmark, the approach achieves state-of-the-art performance, significantly advancing language grounding capabilities in dynamic 4D environments.

Existing 3D perception methods often rely on object-centric modeling or extensive scene-specific training, hindering unified and efficient open-vocabulary reasoning. This work proposes a training-free, unified framework that constructs a hierarchical 3D scene representation by distilling language-aligned Gaussian splats, refines geometry through Gaussian pruning, and aggregates multi-view 2D features via language-guided alignment to produce precise 3D object embeddings. Building upon this representation, the method constructs an open-vocabulary 3D semantic scene graph that jointly models hierarchical semantics and intra- and inter-object relationships, enabling unified reasoning across segmentation, retrieval, and relational understanding. Experiments demonstrate that the approach is both efficient and scalable across multiple tasks.

Traditional geometric models face fundamental limitations in real-time depth perception, occlusion handling, and modeling of textureless surfaces. To address these challenges, this paper presents a systematic review and advancement of deep learning–driven scene understanding for autonomous robots. We propose an end-to-end framework integrating CNN-Transformer hybrid architectures, self-supervised depth estimation, multi-task joint training, and NeRF-enhanced representation learning. The framework significantly improves the synergistic performance of semantic segmentation, 3D reconstruction, and visual SLAM in dynamic, unstructured environments. Key innovations include occlusion-robust dense depth inference and cross-modal semantic-geometric joint representation, which collectively enhance robots’ real-time perception, navigation decision-making, and physical interaction capabilities. This work establishes a unified methodology and scalable technical pathway for learning-based embodied scene understanding.

This work addresses the limitation of existing vision-language models (VLMs) in spatial reasoning, which stems from their lack of explicit three-dimensional scene topology and reliance on non-geometric semantic features. The study is the first to uncover and model the implicit 3D topological structure within VLMs, introducing a theoretically grounded regularization approach. This method employs cross-scenario linear probes to extract latent subspaces and constrains them using Laplacian eigenmaps, Gaussian kernel graphs, and Dirichlet energy. Remarkably, with only 500 steps of fine-tuning on synthetic data, the proposed technique outperforms standard fine-tuning and strong baselines by up to 12.1% on real-world spatial reasoning tasks, substantially enhancing the model’s spatial generalization capabilities.

Existing spatial semantic representations struggle to effectively reason about structured temporal dynamics—such as the periodic movement of household objects—in semi-static environments. This work proposes PredictiveGraphs, a predictive 3D scene graph that integrates spatiotemporal and semantic information by embedding Perpetua* Bayesian filters directly into inter-node relationships, enabling temporal modeling and future prediction of object states. By jointly modeling spatiotemporal-semantic relations and performing recurrent state inference, the approach maintains robustness under distributional shifts. Evaluated over three-week navigation tasks in both simulation and real-world settings—with environmental changes occurring every two hours—the method significantly outperforms current baselines in accurately forecasting the dynamic evolution of the environment.

The field of 3D vision suffers from fragmented data representations, learning paradigms, and benchmarking protocols, leading to a lack of unified understanding regarding efficiency, fidelity, and scalability. This work proposes the first cohesive conceptual framework that integrates geometric representations—such as point clouds, meshes, voxels, and 3D Gaussians—with diverse learning paradigms—including 2D-supervised learning, implicit neural representations, and 4D modeling—and connects them to real-world application scenarios. By constructing a structured knowledge graph of 3D vision, the study systematically relates dataset design, supervision mechanisms, and task requirements, clarifying the trade-offs between efficiency and fidelity and charting pathways for multimodal geometric grounding. This framework offers systematic guidance for reconstruction, generation, and dynamic scene modeling, advancing the field toward a unified and efficient paradigm.