Multi-Camera AR Guidance System for Surgical Instrument Handling and Assembly: Investigating Workload and Efficiency

2026-06-03

AI Summary

This study addresses the high cognitive load imposed on scrub nurses during surgical instrument assembly, particularly when handling unfamiliar instruments. The authors propose a markerless multi-camera 6D pose estimation and unmarked augmented reality (AR) in-situ guidance method that relies solely on synthetic data to train the pose estimation network. Assembly instructions are overlaid via a head-mounted display, and hands-free interaction is enabled through eye-gaze selection and foot-pedal switching. The approach achieves superior 6D pose accuracy compared to existing methods. User studies demonstrate a 21.3% reduction in task time (averaging 4.76 minutes), significantly lower subjective cognitive load—especially benefiting novice nurses—and comparable error rates, thereby confirming its clinical applicability and generalization capability.

Abstract

The handling and assembly of instruments during surgery imposes high cognitive demands on scrub nurses, particularly when instruments are unfamiliar. We present a supporting guidance system for surgical instrumentation that combines multi-camera 6D pose estimation with augmented reality in-situ visualization on a head-mounted display without the requirement for additional markers. Pose estimation and consecutive camera calibration are achieved through known objects. The 6D pose estimation network is trained purely on synthetic data, aiming for better generalizability and real-world applicability. The AR guidance displays tooltip localization cues and step-wise assembly animations. Via gaze-based selection and a foot pedal, users can switch between assembly steps in intraoperative use. In a technical evaluation, our approach outperforms state-of-art 6D pose estimation. A user study with 29 scrub nurses was conducted in a surgical simulation of knee arthroplasty, comparing the system against a paper manual. AR guidance significantly reduced the perceived workload compared. Objectively, AR guidance reduced task completion time by 21.3\% (4.76 minutes). Specifically, scrub nurses less experienced with the instrument set benefited when using the system. Error frequencies were comparable between conditions. Qualitative feedback highlighted improved process clarity, reduced information overload, and perceived independence. To summarize, our marker-free multi-camera AR guidance approach for surgical instruments can, subjectively and objectively, improve intraoperative instrumentation performance, particularly for untrained scrub nurses.

Problem

• surgical instrument handling
• cognitive workload
• scrub nurse
• instrument assembly
• intraoperative efficiency

Innovation

• marker-free
• multi-camera
• 6D pose estimation
• augmented reality
• synthetic data training