ExpOS: Explainable Open-Surgery Skills Assessment Using 3D Hand Reconstruction

2026-05-22

AI Summary

This study addresses the longstanding reliance on expert observation for open surgical skill assessment, which hinders timely and scalable feedback. The authors propose a weakly supervised framework that integrates temporal importance learning with interpretable motion features, leveraging 3D hand reconstruction and instrument detection to capture hand-instrument dynamics. By combining temporal convolutions, attention mechanisms, and global motion statistics, the model automatically predicts surgical skill levels. Notably, it offers multi-level interpretability—identifying critical temporal segments and revealing key motion features driving the assessment. Evaluated on fascial closure videos from 221 medical students, the method achieves strong correlation with expert ratings (r = 0.778, R² = 0.74), demonstrating both effectiveness and scalability.

Abstract

Timely and transparent feedback is essential for effective surgical training, yet current assessment remains dependent on expert observation, limiting scalability and opportunities for autonomous practice. We present ExpOS, an explainable framework for data-driven assessment of open-surgery skills designed to enable automatic, feedback-oriented evaluation. Rather than relying on expert-defined metrics, ExpOS learns discriminative temporal patterns directly from motion data and identifies the segments and behaviors most predictive of skill level. We trained and evaluated the method on 221 videos of medical students performing three open-surgery tasks. Hand poses and tool detections were extracted from each frame to derive kinematic descriptors and global motion statistics. Spatiotemporal hand-tool dynamics were modeled using a temporal convolutional backbone with attention-based pooling to generate frame-level importance maps. These representations were fused with global motion statistics to predict skill level and to provide interpretable feedback. ExpOS provides multi-level explainability by identifying when informative events occur through attention weights and which motion characteristics most influence predictions through global feature analysis. Across tasks, the framework achieved strong correlation with expert ratings, with best performance on fascial closure (r = 0.778, R2 = 0.74). These results demonstrate that combining weakly-supervised temporal importance learning with interpretable motion statistics enables scalable and actionable surgical skill assessment.

Problem

• surgical skills assessment
• explainable AI
• open surgery
• automatic feedback
• motion analysis

Innovation

• Explainable AI
• 3D hand reconstruction
• Surgical skill assessment
• Temporal convolutional network
• Attention-based pooling