SCOPE-4D: Endoscopic 4D Geometry Foundation Models

2026-10-01

AI Summary

This study addresses the scarcity of geometric annotations in endoscopic scenes and the estimation ambiguity arising from the coupling of camera motion and tissue deformation. To tackle these challenges, we propose a geometry foundation model for monocular endoscopic videos. Methodologically, we introduce a Co-Residual Motion (CRM) constraint to decouple and regularize local tissue deformation. Combined with large-scale geometric supervised fine-tuning on the SCOPE-5K dataset, the proposed approach enables joint prediction of camera parameters, dense geometry, and 3D tissue trajectories. Experimental results demonstrate that our model significantly improves geometric reconstruction accuracy and 3D tracking stability across both in-domain and out-of-domain scenarios, while also receiving positive recognition from clinical experts.

Abstract

Geometric understanding supports endoscopic navigation and robotic assistance, but learning reliable endoscopic geometry faces two challenges: scarce geometric annotations and ambiguity between camera motion and tissue deformation. We present SCOPE-4D, an endoscopic 4D geometry foundation model that jointly predicts camera parameters, dense geometry, and 3D tissue trajectories from monocular RGB video in a single forward pass. Our curation and annotation pipeline constructs SCOPE-5K, a collection of approximately 5,000 clips spanning real and synthetic gastrointestinal endoscopy and laparoscopy. The collection provides rich geometric supervision and includes newly collected phantom and real-colonoscopy evaluation sets. Geometric supervised fine-tuning on SCOPE-5K learns endoscopic priors that improve camera and depth estimation. Common--Residual Motion (CRM) further constrains local deformation relative to common tissue movement. Together with geometric supervision, CRM and trajectory supervision further improve camera and depth estimation over geometric fine-tuning alone while enabling dense 3D tissue tracking. Evaluations on public and newly collected benchmarks demonstrate strong in-domain and out-of-domain geometry, superior 3D tracking, and more stable long-sequence colon reconstruction. A blinded user study further supports the perceived reconstruction quality on real clinical video. Together, these results demonstrate the value of large-scale endoscopic supervision and motion constraints for joint geometry estimation and tissue tracking.

Problem

• endoscopic geometry
• geometric annotations
• camera motion
• tissue deformation
• ambiguity

Innovation

• Endoscopic 4D Geometry Foundation Model
• Monocular RGB Video
• Common-Residual Motion (CRM)
• Dense 3D Tissue Tracking
• Geometric Supervised Fine-tuning