The Dresden Dataset for 4D Reconstruction of Non-Rigid Abdominal Surgical Scenes

2026-03-03

AI Summary

This work addresses the lack of high-quality benchmark data for evaluating 4D reconstruction of non-rigid abdominal soft tissue deformation under realistic surgical conditions. To this end, we present the first multimodal dataset comprising synchronized endoscopic videos and ground-truth geometry captured via structured light, encompassing diverse non-rigid deformations, instrument occlusions, and precise camera poses. Data were acquired using a da Vinci Xi endoscope and a Zivid structured-light camera, with multimodal alignment achieved through optical tracking, ICP refinement, and semi-automatic registration. The dataset includes 98 sequences, over 300,000 video frames, and 369 point clouds, and provides instrument masks and rectified images. Crucially, it enables the first quantitative evaluation of both visible and occluded regions, establishing a rigorous benchmark for SLAM, 4D reconstruction, and depth estimation in non-rigid surgical scenarios.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Security and Privacy: Data transparency and provenance
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

The D4D Dataset provides paired endoscopic video and high-quality structured-light geometry for evaluating 3D reconstruction of deforming abdominal soft tissue in realistic surgical conditions. Data were acquired from six porcine cadaver sessions using a da Vinci Xi stereo endoscope and a Zivid structured-light camera, registered via optical tracking and manually curated iterative alignment methods. Three sequence types - whole deformations, incremental deformations, and moved-camera clips - probe algorithm robustness to non-rigid motion, deformation magnitude, and out-of-view updates. Each clip provides rectified stereo images, per-frame instrument masks, stereo depth, start/end structured-light point clouds, curated camera poses and camera intrinsics. In postprocessing, ICP and semi-automatic registration techniques are used to register data, and instrument masks are created. The dataset enables quantitative geometric evaluation in both visible and occluded regions, alongside photometric view-synthesis baselines. Comprising over 300,000 frames and 369 point clouds across 98 curated recordings, this resource can serve as a comprehensive benchmark for developing and evaluating non-rigid SLAM, 4D reconstruction, and depth estimation methods.

Problem

• 4D reconstruction
• non-rigid deformation
• surgical scene
• abdominal soft tissue
• 3D reconstruction

Innovation

• 4D reconstruction
• non-rigid deformation
• surgical scene dataset
• structured-light scanning
• multi-modal registration