PointSea: Point Cloud Completion via Self-structure Augmentation

2025-02-24

AI Summary

Point cloud completion is challenging due to complex structural configurations in missing regions and the absence of texture cues. To address this, we propose PointSea, a global-to-local self-structured enhancement framework. In the global stage, multi-view self-projected depth maps are leveraged to construct robust geometric representations, enhanced by cross-view and cross-modal feature fusion. In the local stage, a structure-conditioned dual-path point generator adaptively refines details based on local geometric priors—such as edges, planar surfaces, and curved regions. PointSea introduces the novel “self-structured enhancement” paradigm, integrating self-supervised geometric similarity modeling with intra- and inter-view feature interaction. Evaluated on benchmarks including ShapeNet, PointSea consistently outperforms state-of-the-art methods, achieving significant improvements in both global shape completeness and local geometric fidelity.

Technology Category

• Computer Vision: Multi-modal Vision
• Search and Optimization: Learning to Search
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search

Abstract

Point cloud completion is a fundamental yet not well-solved problem in 3D vision. Current approaches often rely on 3D coordinate information and/or additional data (e.g., images and scanning viewpoints) to fill in missing parts. Unlike these methods, we explore self-structure augmentation and propose PointSea for global-to-local point cloud completion. In the global stage, consider how we inspect a defective region of a physical object, we may observe it from various perspectives for a better understanding. Inspired by this, PointSea augments data representation by leveraging self-projected depth images from multiple views. To reconstruct a compact global shape from the cross-modal input, we incorporate a feature fusion module to fuse features at both intra-view and inter-view levels. In the local stage, to reveal highly detailed structures, we introduce a point generator called the self-structure dual-generator. This generator integrates both learned shape priors and geometric self-similarities for shape refinement. Unlike existing efforts that apply a unified strategy for all points, our dual-path design adapts refinement strategies conditioned on the structural type of each point, addressing the specific incompleteness of each point. Comprehensive experiments on widely-used benchmarks demonstrate that PointSea effectively understands global shapes and generates local details from incomplete input, showing clear improvements over existing methods.

Problem

• PointSea addresses incomplete 3D point cloud data.
• It uses self-structure augmentation for global-to-local completion.
• PointSea refines shapes based on point-specific structural types.

Innovation

• Self-projected depth images augmentation
• Feature fusion at intra-inter view levels
• Self-structure dual-generator for refinement