3D Reconstruction of the Human Colon fromCapsule Endoscope Video

2024-07-21

Colour and Visual Computing Symposium

AI Summary

Wireless capsule endoscopy (WCE) video-based 3D colonic reconstruction faces challenges from lens distortion, motion artifacts, and sparse camera poses. To address these, we propose an end-to-end SLAM-driven framework: (1) robust pose estimation via improved visual SLAM; (2) multi-frame feature fusion to construct a sparse point cloud, followed by Poisson surface reconstruction for coherent colonic geometry; and (3) a parametric virtual gastrointestinal phantom model enabling controlled distortion simulation, forming the first geometric error quantification framework tailored for WCE. This work achieves, for the first time, full-length 3D colonic reconstruction directly from real clinical WCE videos—demonstrating the feasibility of SLAM and Poisson reconstruction in this domain. The resulting models ensure measurable structural fidelity and full reproducibility, establishing an interpretable, verifiable AI-assisted diagnostic foundation for early colorectal cancer screening.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

As the number of people affected by diseases in the gastrointestinal system is ever-increasing, a higher demand on preventive screening is inevitable. This will significantly increase the workload on gastroenterologists. To help reduce the workload, tools from computer vision may be helpful. In this paper, we investigate the possibility of constructing 3D models of whole sections of the human colon using image sequences from wireless capsule endoscope video, providing enhanced viewing for gastroenterologists. As capsule endoscope images contain distortion and artifacts non-ideal for many 3D reconstruction algorithms, the problem is challenging. However, recent developments of virtual graphics-based models of the human gastrointestinal system, where distortion and artifacts can be enabled or disabled, makes it possible to ``dissect'' the problem. The graphical model also provides a ground truth, enabling computation of geometric distortion introduced by the 3D reconstruction method. In this paper, most distortions and artifacts are left out to determine if it is feasible to reconstruct whole sections of the human gastrointestinal system by existing methods. We demonstrate that 3D reconstruction is possible using simultaneous localization and mapping. Further, to reconstruct the gastrointestinal wall surface from resulting point clouds, varying greatly in density, Poisson surface reconstruction is a good option. The results are promising, encouraging further research on this problem.

Problem

• Reconstruct 3D colon models from capsule endoscope videos
• Address distortion and artifacts in endoscope images
• Evaluate feasibility using SLAM and Poisson surface reconstruction

Innovation

• 3D reconstruction from capsule endoscope video
• Using virtual graphics-based models for validation
• Poisson surface reconstruction for sparse point clouds