The Reconstructive Metaverse – Collaboration in Real-Time Shared Mixed Reality Environments for Microsurgical Reconstruction

Necker FN, Cholok DJ, Shaheen MS, Fischer MJ, Gifford K, El Chemaly T, Leuze CW, Scholz M, Daniel BL, Momeni A (2024)

Publication Type: Journal article

Publication year: 2024

Journal

Surgical Innovation SAGE Publications (UK and US)

DOI: 10.1177/15533506241262946

Abstract

Plastic surgeons routinely use 3D-models in their clinical practice, from 3D-photography and surface imaging to 3D-segmentations from radiological scans. However, these models continue to be viewed on flattened 2D screens that do not enable an intuitive understanding of 3D-relationships and cause challenges regarding collaboration with colleagues. The Metaverse has been proposed as a new age of applications building on modern Mixed Reality headset technology that allows remote collaboration on virtual 3D-models in a shared physical-virtual space in real-time. We demonstrate the first use of the Metaverse in the context of reconstructive surgery, focusing on preoperative planning discussions and trainee education. Using a HoloLens headset with the Microsoft Mesh application, we performed planning sessions for 4 DIEP-flaps in our reconstructive metaverse on virtual patient-models segmented from routine CT angiography. In these sessions, surgeons discuss perforator anatomy and perforator selection strategies whilst comprehensively assessing the respective models. We demonstrate the workflow for a one-on-one interaction between an attending surgeon and a trainee in a video featuring both viewpoints as seen through the headset. We believe the Metaverse will provide novel opportunities to use the 3D-models that are already created in everyday plastic surgery practice in a more collaborative, immersive, accessible, and educational manner.

Authors with CRIS profile

Michael Scholz Lehrstuhl für Funktionelle und Klinische Anatomie

Involved external institutions

Stanford University US United States (USA) (US)

How to cite

APA:

Necker, F.N., Cholok, D.J., Shaheen, M.S., Fischer, M.J., Gifford, K., El Chemaly, T.,... Momeni, A. (2024). The Reconstructive Metaverse – Collaboration in Real-Time Shared Mixed Reality Environments for Microsurgical Reconstruction. Surgical Innovation. https://doi.org/10.1177/15533506241262946

MLA:

Necker, Fabian N., et al. "The Reconstructive Metaverse – Collaboration in Real-Time Shared Mixed Reality Environments for Microsurgical Reconstruction." Surgical Innovation (2024).

BibTeX: Download