HIGS: Hierarchical Implicit Grids for Joint Geometric and Semantic Scene Understanding

2026-09-29

AI Summary

This study addresses the limitations of existing neural implicit representations, which lack semantic information and struggle to scale to large environments. To this end, we propose a hierarchical neural field method that enables efficient joint modeling of geometry and semantics through multi-resolution submaps. A unified query-decoding mechanism and feature encoder are designed to align and fuse these submaps within an implicit space, thereby eliminating accumulated drift. Furthermore, the approach incorporates hierarchical optimization, vision-language latent feature embedding, and signed distance field construction. Experimental results demonstrate that the proposed method significantly improves both computational and memory efficiency while maintaining high estimation accuracy, ultimately endowing robots with robust spatial perception capabilities in large-scale environments.

Abstract

Neural implicit representations have had a significant impact on scene reconstruction by enabling robots to build continuous, differentiable, and high-fidelity 3D maps. Most existing works focus on geometric reconstruction and lack semantic information for high-level spatial understanding and task planning. Also, as the scale and complexity of the environment increase, neural representations face the challenge of maintaining computational efficiency in back-end optimization. To resolve these two challenges, we introduce a hierarchical neural field that leverages multiresolution submaps to achieve an efficient and scalable implicit representation, and a unified query and decoding mechanism to support both geometric and semantic features. More specifically, the learnable map features can be converted to the output with the query and decoding process for both training and inference. For large-scale representation, we decompose a scene into overlapping submaps and do hierarchical optimization within each local submap, thus enabling scalable computation. To further improve efficiency, we design feature encoders that predict initial hierarchical grid features to substantially reduce the time needed to optimize the submap features from scratch. To correct estimation drift among submaps, we align and fuse them entirely within the implicit feature space, leading to substantial acceleration by avoiding the need to decode the final output. Building upon this efficient hierarchical representation, we embed both geometric features and vision-language latent features into the map, and demonstrate it on both Signed Distance Field (SDF) construction and open-vocabulary object grounding. Our approach significantly improves computation and memory efficiency, maintains high estimation accuracy, and endows the robot with spatial awareness on large-scale real-world benchmarks.

Problem

• Neural implicit representations
• Semantic scene understanding
• Computational efficiency
• Scalability
• Geometric reconstruction

Innovation

• Hierarchical Implicit Grids
• Neural Implicit Representations
• Joint Geometric and Semantic Understanding
• Scalable Scene Reconstruction
• Open-vocabulary Object Grounding