Empowering the OR: Assistance for Surgical Navigation and Robotics

During the PROFICIENCY project, Tim Flückiger et al. developed and evaluated a novel, fast, and fully automatic calibration method designed to eliminate the need for operator intervention or specialized expertise, thereby reducing the sim-to-real gap in surgical navigation and robotic surgery.

The automated and highly accurate external camera calibration approach for multi-camera systems used in 3D Surgical Scene Reconstruction (3D-SSR) has proven to be an effective alternative to traditional, operator-dependent, marker-based techniques.

While some limitations remain to be addressed, this study marks an important step toward providing hospital staff with more efficient and reliable technological assistance.

Figure: Examples of camera viewpoints used for 3D surgical scene reconstruction. From left to right: far-field camera C1 (ceiling-mounted GoPro), near-field camera C2 (surgical lamp-mounted GoPro), and near-field camera C3 (ceiling-mounted Canon CR-N300), zoomed in on the surgical field. Bottom: Inter-view point correspondences provided by our multi-camera calibration system, observed from the same viewpoints. From left to right: all correspondences seen by C1 (blue, green, and red), correspondences between C1 and C2 (green), and correspondences between C1 and C3 (red). The surgeon’s body and hand poses, represented by blue lines in the first row, are successfully reconstructed using the estimated camera poses.