Point Cloud Segmentation for Autonomous Clip Positioning in Laparoscopic Cholecystectomy on a Phantom

2026-06-10

AI Summary

This work addresses the challenges of data scarcity and procedural safety in laparoscopic cholecystectomy by presenting the first high-precision, interpretable autonomous clip placement system implemented on a physical model. The method leverages a single-camera setup to generate colorless point clouds, from which target locations are extracted via point cloud segmentation and spline interpolation, with optional manual correction. By combining synthetic data pretraining with two novel point cloud data augmentation strategies, the system achieves clinical-grade accuracy using only 60 real annotated samples. Real-world robotic experiments demonstrate a target localization accuracy of 0.75 mm (95% confidence) and a 100% success rate in autonomous clip placement, while providing verifiable visualizations of the motion planning process.

Abstract

High-risk applications in robotics, such as robot-assisted surgery, present unique challenges. These systems must be both highly precise and interpretable in order to be deployed in environments with very low tolerance for error or unsafe exploration. We present the first robotic system to demonstrate autonomous clip positioning on a physical phantom in laparoscopic surgery, one of the most common interventions in general surgery. After segmentation of a colorless point cloud from a single camera, target positions for the clips are extracted using spline interpolation, and can then be adjusted by the human operator. The segmentation model is trained on only 60 hand-labeled real point clouds, reflecting data scarcity in the surgical domain. We overcome this with a combination of pre-training on 128,000 synthetic point clouds and two novel data augmentation techniques. The motion of the end-effector to each target is visualized for the operator, satisfying the unique motion constraints of minimally-invasive surgery while ensuring that the robot's actions are verifiable and interpretable. In real robot experiments, our system localizes targets with the required precision of 0.75mm at a 95% success rate and executes autonomous clip positioning with a 100% success rate. We provide insights that are applicable to many other surgical and non-surgical tasks that require identifying and navigating to a precise target. Source code and project page: https://github.com/balazsgyenes/kirurc

Problem

• Point Cloud Segmentation
• Autonomous Clip Positioning
• Laparoscopic Cholecystectomy
• Data Scarcity
• Robot-assisted Surgery

Innovation

• point cloud segmentation
• autonomous clip positioning
• surgical robotics
• data augmentation
• interpretable robot motion