PICO: Projection-Informed Consistency Optimisation for 6DoF Surgical Tool Pose Estimation

2026-09-25

AI Summary

This study addresses the limitations of existing approaches for 6DoF pose estimation of surgical instruments, specifically the error accumulation inherent in kinematic methods and the poor real-time performance and reliance on external markers characteristic of vision-based methods. To overcome these challenges, this work proposes PICO, an end-to-end model built upon a multi-task learning architecture that jointly predicts segmentation, depth, and pose parameters. Notably, PICO introduces a geometry-aware proxy task and enforces geometric consistency through projection and point-to-point losses. Evaluated on the SurgRIPE dataset, the proposed method achieves the second-highest rotational accuracy while demonstrating competitive translational performance. Furthermore, PICO significantly enhances both the robustness and precision of pose estimation under occluded conditions, offering a promising solution for markerless, real-time surgical instrument tracking.

Abstract

Purpose: Accurate 6 DoF pose estimation of surgical tools is critical for automa- tion, robotic proprioception, and safe interaction with the tissue operated on. Kinematics-based approaches suffer from accumulated errors due to the cable- driven nature of robotic arms, while vision-based methods often rely on external markers or trackers. Although more recent vision-based advances have been pro- posed, these two-stage pose estimation methods often lack real-time robustness due to accumulated errors and computational overhead. Methods: We propose a novel end-to-end trainable model, PICO. Our model employs a multi-task learning architecture to predict segmentation and depth maps, alongside regression of translation and rotation parameters. We define two proxy tasks that enforce geometric consistency in both 2D and 3D spaces, improving accuracy and robustness. For this, we propose a projection loss, and a point-to-point loss. Results: We evaluate our method on the SurgRIPE dataset, benchmarking its performance against state-of-the-art approaches using standard 6DoF pose esti- mation metrics. Our results demonstrate consistently strong performance across all four subsets, specifically in rotation, ranking second even under occlusion. It also demonstrates comparable translational performance, remaining competitive, especially in occluded cases. Conclusion: PICO demonstrates the effectiveness of multi-task learning and geometry-aware proxy tasks for robust and reliable surgical tool pose estimation, especially in occluded scenarios, highlighting potential for future applications.

Problem

• 6DoF pose estimation
• surgical tools
• accumulated errors
• real-time robustness
• occlusion

Innovation

• 6DoF Pose Estimation
• Multi-task Learning
• Geometric Consistency
• End-to-End Training
• Projection Loss