AI-based single-shot structured-light depth reconstruction for real-time laparoscopic surgical guidance

2026-08-05

AI Summary

This work addresses the challenge of depth reconstruction in compact laparoscopic systems, where conventional structured light methods—requiring multi-frame acquisition and precise projector-camera synchronization—are impractical. The authors propose a synchronization-free, single-shot depth estimation framework that integrates passive binary LED structured light, a VQ-VAE with a discrete latent space prior, and a custom U-Net depth prediction head to enable real-time, video-rate depth inference from a single frame in a dual-channel laparoscope. Notably, this is the first study to incorporate VQ-VAE latent representations into endoscopic depth reconstruction, achieving high-fidelity depth recovery without explicit segmentation and enabling deployment on low-complexity hardware. Experiments demonstrate state-of-the-art performance with a mean absolute error of 3.70 mm, 96.2% accuracy under the δ<1.1 metric, and an inference speed of 26.0 Hz, significantly outperforming existing baselines.

Abstract

Significance. Accurate intraoperative depth perception is important for autonomous and semi-autonomous robotic laparoscopic surgery. Conventional fringe projection profilometry can achieve millimeter-scale accuracy but often requires multi-shot acquisition, digital-micromirror-device projection, and projector-camera synchronization, complicating integration into compact laparoscopic systems. Aim. To develop a synchronization-free, single-shot depth-sensing platform using a passive LED-illuminated binary mask and a VQ-VAE prior with a custom U-Net depth head. Approach. A compact projection module was coupled to one channel of a dual-channel laparoscope, while the second channel imaged the fringe-illuminated target. A Zivid 3D camera acquired reference depth for 722 paired phantom images. Zivid depth maps were reprojected into the SSLE image frame for supervised training and evaluation. The VQ-VAE encoded each input into a discrete latent representation, and a latent-space U-Net predicted depth without a separate mask-prediction branch. Results. Using a fixed train/validation/test split, the proposed model achieved an MAE of 3.70 mm, AbsRel of 0.0326, delta=1.1 accuracy of 0.962, and delta=1.1^2 accuracy of 0.970. It achieved lower MAE than the dual U-Net MaskNet + DepthNet baseline and outperformed off-the-shelf monocular depth models in MAE, AbsRel, and threshold accuracy. The pipeline operated at 26.0 Hz over 301 consecutive frames on an NVIDIA A100 GPU. Conclusions. The LED-illuminated binary-pattern platform with latent-space depth reconstruction enables synchronization-free, video-rate endoscopic depth estimation. Results demonstrate Zivid-referenced phantom reconstruction without an explicit segmentation stage, while emphasizing the importance of dataset size and SSLE-Zivid calibration accuracy.

Problem

• depth reconstruction
• single-shot
• laparoscopic surgery
• real-time
• structured light

Innovation

• single-shot depth reconstruction
• VQ-VAE latent space
• synchronization-free structured light
• laparoscopic depth estimation
• binary mask projection