Towards unsupervised learning for instrument segmentation in robotic surgery with cycle-consistent adversarial networks

Pakhomov D, Shen W, Navab N (2020)


Publication Type: Conference contribution

Publication year: 2020

Journal

Publisher: Institute of Electrical and Electronics Engineers Inc.

Pages Range: 8499-8504

Conference Proceedings Title: IEEE International Conference on Intelligent Robots and Systems

Event location: Las Vegas, NV, USA

ISBN: 9781728162126

DOI: 10.1109/IROS45743.2020.9340816

Abstract

Surgical tool segmentation in endoscopic images is an important problem: it is a crucial step towards full instrument pose estimation and it is used for integration of pre- and intra-operative images into the endoscopic view. While many recent approaches based on convolutional neural networks have shown great results, a key barrier to progress lies in the acquisition of a large number of manually-annotated images which is necessary for an algorithm to generalize and work well in diverse surgical scenarios. Unlike the surgical image data itself, annotations are difficult to acquire and may be of variable quality. On the other hand, synthetic annotations can be automatically generated by using forward kinematic model of the robot and CAD models of tools by projecting them onto an image plane. Unfortunately, this model is very inaccurate and cannot be used for supervised learning of image segmentation models. Since generated annotations will not directly correspond to endoscopic images due to errors, we formulate the problem as an unpaired image-to-image translation where the goal is to learn the mapping between an input endoscopic image and a corresponding annotation using an adversarial model. Our approach allows to train image segmentation models without the need to acquire expensive annotations and can potentially exploit large unlabeled endoscopic image collection outside the annotated distributions of image/annotation data. We test our proposed method on Endovis 2017 challenge dataset and show that it is competitive with supervised segmentation methods.

Involved external institutions

How to cite

APA:

Pakhomov, D., Shen, W., & Navab, N. (2020). Towards unsupervised learning for instrument segmentation in robotic surgery with cycle-consistent adversarial networks. In IEEE International Conference on Intelligent Robots and Systems (pp. 8499-8504). Las Vegas, NV, USA: Institute of Electrical and Electronics Engineers Inc..

MLA:

Pakhomov, Daniil, Wei Shen, and Nassir Navab. "Towards unsupervised learning for instrument segmentation in robotic surgery with cycle-consistent adversarial networks." Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2020, Las Vegas, NV, USA Institute of Electrical and Electronics Engineers Inc., 2020. 8499-8504.

BibTeX: Download