Surgical Re-enactment for Operating Room Workflow Datasets

2026-07-27

AI Summary

Acquiring high-quality surgical workflow data that satisfies ethical and sterility requirements while encompassing multiple roles and viewpoints remains challenging in real operating rooms. To address this, this work proposes the first end-to-end replay methodology tailored for robot-assisted ophthalmic surgery. By formalizing surgical workflows, reconstructing the operating room environment, systematically training role-specific participants, and conducting iterative recording sessions, the approach yields a reproducible and annotatable high-fidelity dataset. This methodology overcomes the inherent limitations of in-situ data collection, successfully generating structured data suitable for activity recognition, scene graph construction, and procedural modeling. Furthermore, it establishes a reusable, standardized data acquisition paradigm with potential applicability across diverse surgical domains.

Abstract

The introduction of new technologies, such as surgical robots, is driving the vision of a connected, smart operating room (OR). However, realizing this vision requires a deep understanding of surgical workflows, which relies on realistic datasets capturing the actions of all OR personnel from both full room and surgical field perspectives. Acquiring such data in real ORs is prohibitively challenging due to factors such as ethics committee approvals, limited space for camera installation, and sterility regulations preventing the use of tracking markers. We present a step-by-step methodology for re-enacting complete surgical procedures in a reconstructed OR. This approach enables the creation of repeatable and annotatable workflow datasets for training activity recognition models, generating scene graphs, and formalizing surgical process models. Developed for robot-assisted ophthalmic surgery, our methodology combines expert consultation, structured workflow formalization, OR reconstruction, role-based training, real OR observation, and iterative recording with post-take debriefing. We provide concrete recommendations to allow other research groups to seamlessly adopt this methodology for their own surgical domains.

Problem

• surgical workflow
• operating room
• dataset acquisition
• data annotation
• surgical re-enactment

Innovation

• Surgical Re-enactment
• Operating Room Workflow
• Dataset Generation
• Activity Recognition
• Surgical Process Modeling