From Clinical Process Optimization to Precision Surgery

During the course of PROFICIENCY, Jonas Hein et al. have demonstrated that marker-less tracking of surgical instruments is becoming a feasible alternative to existing marker-based systems.

Traditional computer vision is increasingly leveraged in the surgical domain. A particular focus in computer-assisted surgery is to replace marker-based tracking systems for instrument localization with pure image-based 6DoF pose estimation using deep-learning methods. The marker-based navigation systems are limited in their applicability while the marker-less approaches have significant potential to seamlessly integrate into the surgical workflow and considerably reduce logistics and calibration overhead.

Figure: Qualitative Comparison of the best single-view, two-view, and five-view baselines on the OR-X bright subset. We superimpose the ground truth pose in green in the second row and pose estimates in orange in the following rows. Yellow triangles in the top-left image corners indicate that the frame was not part of the input to the pose estimation method. Note that the single-view pose estimate has a great visual overlap on the input image, but a significant error when viewed from other perspectives due the depth ambiguity. Source: https://doi.org/10.1016/j.media.2025.103613

Keywords: Multi-view RGB-D video dataset; Marker-less tracking; Surgical instruments; Object pose estimation; Surgical navigation; Deep Learning