A real-time interactive augmented reality depth estimation technique for surgical robotics

Kalia M, Navab N, Salcudean T (2019)

Publication Type: Conference contribution

Publication year: 2019

Journal

Proceedings - IEEE International Conference on Robotics and Automation Institute of Electrical and Electronics Engineers

Publisher: Institute of Electrical and Electronics Engineers Inc.

Book Volume: 2019-May

Pages Range: 8291-8297

Conference Proceedings Title: Proceedings - IEEE International Conference on Robotics and Automation

Event location: Montreal, QC, CAN

ISBN: 9781538660263

DOI: 10.1109/ICRA.2019.8793610

Abstract

Augmented reality (AR) is a promising technology where the surgeon can see the medical abnormality in the context of the patient. It makes the anatomy of interest visible to the surgeon which otherwise is not visible. It can result in better surgical precision and therefore, potentially better surgical outcomes and faster recovery times. Despite these benefits, the current AR systems suffer from two major challenges; first, incorrect depth perception and, second, the lack of suitable evaluation systems. Therefore, in the current paper we addressed both of these problems. We proposed a color depth encoding (CDE) technique to estimate the distance between the tumor and the tissue surface using a surgical instrument. We mapped the distance between the tumor and the tissue surface to the blue-red color spectrum. For evaluation and interaction with our AR technique, we propose to use a virtual surgical instrument method using the CAD model of the instrument. The users were asked to reach the judged distance in the surgical field using the virtual tool. Realistic tool movement was simulated by collecting the forward kinematics joint encoder data. The results showed significant improvement in depth estimation, time for task completion and confidence, using our CDE technique with and without stereo versus other two cases, that are, Stereo-No CDE and No Stereo-No CDE.

Involved external institutions

University of British Columbia CA Canada (CA) Technische Universität München (TUM) DE Germany (DE)

How to cite

APA:

Kalia, M., Navab, N., & Salcudean, T. (2019). A real-time interactive augmented reality depth estimation technique for surgical robotics. In Proceedings - IEEE International Conference on Robotics and Automation (pp. 8291-8297). Montreal, QC, CAN: Institute of Electrical and Electronics Engineers Inc..

MLA:

Kalia, M., N. Navab, and T. Salcudean. "A real-time interactive augmented reality depth estimation technique for surgical robotics." Proceedings of the 2019 International Conference on Robotics and Automation, ICRA 2019, Montreal, QC, CAN Institute of Electrical and Electronics Engineers Inc., 2019. 8291-8297.

BibTeX: Download