Automatic Re-orientation of 3D Echocardiographic Images in Virtual Reality Using Deep Learning
Munroe L, Sajith G, Lin E, Bhattacharya S, Pushparajah K, Simpson J, Schnabel JA, Wheeler G, Gomez A, Deng S (2021)
Publication Type: Conference contribution
Publication year: 2021
Journal
Publisher: Springer Science and Business Media Deutschland GmbH
Book Volume: 12722 LNCS
Pages Range: 177-188
Conference Proceedings Title: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Event location: Virtual, Online
ISBN: 9783030804312
DOI: 10.1007/978-3-030-80432-9_14
Abstract
In 3D echocardiography (3D echo), the image orientation varies depending on the position and direction of the transducer during examination. As a result, when reviewing images the user must initially identify anatomical landmarks to understand image orientation – a potentially challenging and time-consuming task. We automated this initial step by training a deep residual neural network (ResNet) to predict the rotation required to re-orient an image to the standard apical four-chamber view). Three data pre-processing strategies were explored: 2D, 2.5D and 3D. Three different loss function strategies were investigated: classification of discrete integer angles, regression with mean absolute angle error loss, and regression with geodesic loss. We then integrated the model into a virtual reality application and aligned the re-oriented 3D echo images with a standard anatomical heart model. The deep learning strategy with the highest accuracy – 2.5D classification of discrete integer angles – achieved a mean absolute angle error on the test set of 9.0∘. This work demonstrates the potential of artificial intelligence to support visualisation and interaction in virtual reality.
Involved external institutions
United Kingdom (GB)How to cite
APA:
Munroe, L., Sajith, G., Lin, E., Bhattacharya, S., Pushparajah, K., Simpson, J.,... Deng, S. (2021). Automatic Re-orientation of 3D Echocardiographic Images in Virtual Reality Using Deep Learning. In Bartłomiej W. Papież, Mohammad Yaqub, Jianbo Jiao, Ana I. Namburete, J. Alison Noble (Eds.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 177-188). Virtual, Online: Springer Science and Business Media Deutschland GmbH.
MLA:
Munroe, Lindsay, et al. "Automatic Re-orientation of 3D Echocardiographic Images in Virtual Reality Using Deep Learning." Proceedings of the 25th Annual Conference on Medical Image Understanding and Analysis, MIUA 2021, Virtual, Online Ed. Bartłomiej W. Papież, Mohammad Yaqub, Jianbo Jiao, Ana I. Namburete, J. Alison Noble, Springer Science and Business Media Deutschland GmbH, 2021. 177-188.
BibTeX: Download