NoKSR: Kernel-Free Neural Surface Reconstruction via Point Cloud Serialization

2025-02-18

AI Summary

Traditional sparse-grid and kernel-based methods suffer from degraded performance in large-scale outdoor point cloud surface reconstruction due to sparse sampling. To address this, we propose a kernel-free neural reconstruction paradigm. First, PointTransformerV3 serializes unordered point clouds into locality-preserving token sequences. Second, a multi-scale token aggregation mechanism is introduced to mitigate neighborhood misclassification induced by serialization. Finally, a lightweight Signed Distance Function (SDF) regression head directly predicts SDF values for query points in 3D space. Our method eliminates explicit voxelization and kernel functions, achieving significant improvements in both accuracy and efficiency. It attains state-of-the-art performance across multiple benchmarks—yielding higher reconstruction fidelity and reducing inference latency by 50%—particularly excelling on sparse outdoor point clouds where it substantially outperforms existing sparse-grid approaches. The framework is conceptually simple, easy to implement, and demonstrates strong generalization capability.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Kernel Methods
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

We present a novel approach to large-scale point cloud surface reconstruction by developing an efficient framework that converts an irregular point cloud into a signed distance field (SDF). Our backbone builds upon recent transformer-based architectures (i.e., PointTransformerV3), that serializes the point cloud into a locality-preserving sequence of tokens. We efficiently predict the SDF value at a point by aggregating nearby tokens, where fast approximate neighbors can be retrieved thanks to the serialization. We serialize the point cloud at different levels/scales, and non-linearly aggregate a feature to predict the SDF value. We show that aggregating across multiple scales is critical to overcome the approximations introduced by the serialization (i.e. false negatives in the neighborhood). Our frameworks sets the new state-of-the-art in terms of accuracy and efficiency (better or similar performance with half the latency of the best prior method, coupled with a simpler implementation), particularly on outdoor datasets where sparse-grid methods have shown limited performance.

Problem

• Kernel-free surface reconstruction
• Point cloud serialization
• Signed distance field prediction

Innovation

• Transformer-based point cloud serialization
• Multi-scale SDF prediction
• Efficient neighbor aggregation