Dense 3D Displacement Estimation for Landslide Monitoring via Fusion of TLS Point Clouds and Embedded RGB Images

2025-06-19

AI Summary

Existing landslide monitoring methods rely on single-modality point clouds, yielding sparse and non-full-3D displacement estimates, which hinder high-fidelity 3D displacement field reconstruction. To address this, we propose a hierarchical coarse-to-fine framework integrating terrestrial laser scanning (TLS) point clouds with synchronized RGB imagery. Our method introduces the first geometric–photometric dual-modality patch matching scheme and a geometric-consistency-driven rigid-body transformation estimation mechanism. It jointly incorporates TLS registration, ResNet-18-based multi-view feature extraction, PatchMatch optimization, RANSAC validation, and local rigid-body solving. Evaluated on real-world landslide data, it achieves up to 97% spatial coverage and sub-resolution accuracy—displacement errors of 0.15 m/0.25 m versus total station/GNSS and 0.07 m/0.20 m versus manual annotations—significantly outperforming TLS’s native resolution (0.08 m/0.30 m) and baseline methods (e.g., F2S3). This work establishes the first end-to-end TLS pipeline enabling high-coverage, full-3D, and high-accuracy displacement reconstruction.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Data transparency and provenance
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Landslide monitoring is essential for understanding geohazards and mitigating associated risks. However, existing point cloud-based methods typically rely on either geometric or radiometric information and often yield sparse or non-3D displacement estimates. In this paper, we propose a hierarchical partition-based coarse-to-fine approach that fuses 3D point clouds and co-registered RGB images to estimate dense 3D displacement vector fields. We construct patch-level matches using both 3D geometry and 2D image features. These matches are refined via geometric consistency checks, followed by rigid transformation estimation per match. Experimental results on two real-world landslide datasets demonstrate that our method produces 3D displacement estimates with high spatial coverage (79% and 97%) and high accuracy. Deviations in displacement magnitude with respect to external measurements (total station or GNSS observations) are 0.15 m and 0.25 m on the two datasets, respectively, and only 0.07 m and 0.20 m compared to manually derived references. These values are below the average scan resolutions (0.08 m and 0.30 m). Our method outperforms the state-of-the-art method F2S3 in spatial coverage while maintaining comparable accuracy. Our approach offers a practical and adaptable solution for TLS-based landslide monitoring and is extensible to other types of point clouds and monitoring tasks. Our example data and source code are publicly available at https://github.com/zhaoyiww/fusion4landslide.

Problem

• Estimates dense 3D landslide displacement via TLS and RGB fusion
• Improves sparse or non-3D results from existing point cloud methods
• Achieves high spatial coverage and accuracy in real-world datasets

Innovation

• Fuses TLS point clouds and RGB images
• Hierarchical coarse-to-fine displacement estimation
• Patch-level matches with geometric consistency