Vis2Reg: Visibility-Aware Landmark-Free Geometric 3D--2D Registration for Liver Laparoscopy

2026-07-20

AI Summary

This work addresses the challenge of registering preoperative 3D models to intraoperative 2D images in laparoscopic liver surgery, where severe occlusions, limited field of view, and the absence of ground-truth 3D supervision hinder accurate alignment. To overcome these issues, we propose a visibility-aware, markerless 3D–2D self-supervised registration framework. Our method leverages intraoperative segmentation masks to constrain deformations to visible regions, combining rigid initialization with an implicit neural deformation field for stable alignment. We introduce a novel mask-based visibility-aware self-supervision signal, enhanced by differentiable point rasterization and mask-guided back-projection, which significantly improves robustness under extreme partial visibility. Evaluated on real intraoperative data, our approach achieves a Dice score of 92.6% and a Chamfer distance of 1.43 mm, with a per-frame inference time of only 111 ms, demonstrating both high accuracy and clinical practicality.

Abstract

Accurate 3D--2D liver registration, which aligns preoperative 3D models to partial, view-dependent intraoperative surface observations, is critical for AR-guided laparoscopic surgery but remains challenging due to severe occlusion, limited visibility, and the lack of 3D ground-truth supervision. Existing landmark-free approaches perform partial-to-complete geometric alignment, yet robust self-supervision under extreme partial visibility remains difficult. We propose Vis2Reg, a visibility-aware registration framework that explicitly constrains deformation using mask-consistent visible regions. We introduce a visibility-aware self-supervision that derives a visible-domain 3D supervision signal from intraoperative masks, enabled by differentiable point rasterization and mask-guided back-projection. This formulation improves robustness under severe occlusion while maintaining fully self-supervised learning. Vis2Reg combines a robust geometric rigid initialization module with an implicit neural deformation field for stable alignment. Vis2Reg achieves a Dice score of 92.6\% and a Chamfer Distance of 1.43 mm on real intraoperative datasets, with 111 ms per-frame inference time, demonstrating both accuracy and practical efficiency.

Problem

• 3D--2D registration
• liver laparoscopy
• occlusion
• limited visibility
• landmark-free

Innovation

• visibility-aware
• self-supervised registration
• landmark-free
• differentiable rasterization
• implicit deformation field