Acquiring Submillimeter-Accurate Multi-Task Vision Datasets for Computer-Assisted Orthopedic Surgery

2025-01-26

AI Summary

Current orthopedic surgical navigation and 3D reconstruction are hindered by the absence of publicly available 3D vision datasets with sub-millimeter accuracy. To address this, we introduce the first open-source, multi-task vision dataset specifically designed for open orthopedic surgery. Built upon an ex vivo porcine spinal model and acquired in a real operating room environment, the dataset features end-to-end ground-truth calibration integrating high-precision 3D scanning, multi-view camera pose estimation, and optical scene registration. This yields jointly annotated intraoperative RGB images and accurate 3D mesh reconstructions—with a root-mean-square (RMS) 3D localization error of 0.35 mm and a spatial resolution of 0.1 mm. The dataset enables training and evaluation of markerless navigation and high-fidelity 3D reconstruction algorithms, thereby establishing a new accuracy benchmark and significantly enhancing generalization capability for surgical vision models.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: Localization, Mapping, and Navigation
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Advances in computer vision, particularly in optical image-based 3D reconstruction and feature matching, enable applications like marker-less surgical navigation and digitization of surgery. However, their development is hindered by a lack of suitable datasets with 3D ground truth. This work explores an approach to generating realistic and accurate ex vivo datasets tailored for 3D reconstruction and feature matching in open orthopedic surgery. A set of posed images and an accurately registered ground truth surface mesh of the scene are required to develop vision-based 3D reconstruction and matching methods suitable for surgery. We propose a framework consisting of three core steps and compare different methods for each step: 3D scanning, calibration of viewpoints for a set of high-resolution RGB images, and an optical-based method for scene registration. We evaluate each step of this framework on an ex vivo scoliosis surgery using a pig spine, conducted under real operating room conditions. A mean 3D Euclidean error of 0.35 mm is achieved with respect to the 3D ground truth. The proposed method results in submillimeter accurate 3D ground truths and surgical images with a spatial resolution of 0.1 mm. This opens the door to acquiring future surgical datasets for high-precision applications.

Problem

• 3D visual dataset
• Orthopedic surgery navigation
• High precision

Innovation

• 3D Scanning
• Optical Scene Registration
• High-Precision Data Generation