GBOT: Graph-Based 3D Object Tracking for Augmented Reality-Assisted Assembly Guidance

Li S, Schieber H, Corell N, Egger B, Kreimeier J, Roth D (2024)

Publication Language: English

Publication Type: Conference contribution

Publication year: 2024

Publisher: Institute of Electrical and Electronics Engineers Inc.

Pages Range: 513-523

Conference Proceedings Title: Proceedings - 2024 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2024

Event location: Orlando, FL US

ISBN: 9798350374025

DOI: 10.1109/VR58804.2024.00072

Abstract

Guidance for assemblable parts is a promising field for augmented reality. Augmented reality assembly guidance requires 6D object poses of target objects in real time. Especially in time-critical medical or industrial settings, continuous and markerless tracking of individual parts is essential to visualize instructions superimposed on or next to the target object parts. In this regard, occlusions by the user's hand or other objects and the complexity of different assembly states complicate robust and real-time markerless multi-object tracking. To address this problem, we present Graph-based Object Tracking (GBOT), a novel graph-based single-view RGB-D tracking approach. The real-time markerless multi-object tracking is initialized via 6D pose estimation and updates the graph-based assembly poses. The tracking through various assembly states is achieved by our novel multi-state assembly graph. We update the multi-state assembly graph by utilizing the relative poses of the individual assembly parts. Linking the individual objects in this graph enables more robust object tracking during the assembly process. For evaluation, we introduce a synthetic dataset of publicly available and 3D printable assembly assets as a benchmark for future work. Quantitative experiments in synthetic data and further qualitative study in real test data show that GBOT can outperform existing work towards enabling context-aware augmented reality assembly guidance. Dataset and code will be made publically available.∗∗∗∗https://github.com/roth-hex-lab/gbot

Authors with CRIS profile

Shiyu Li Juniorprofessur für Human-Centered Computing and Extended Reality Hannah Schieber Juniorprofessur für Human-Centered Computing and Extended Reality Bernhard Egger Professur für Cognitive Computer Vision (Stiftungsprofessur)

Involved external institutions

Technische Universität München (TUM) DE Germany (DE)

How to cite

APA:

Li, S., Schieber, H., Corell, N., Egger, B., Kreimeier, J., & Roth, D. (2024). GBOT: Graph-Based 3D Object Tracking for Augmented Reality-Assisted Assembly Guidance. In Proceedings - 2024 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2024 (pp. 513-523). Orlando, FL, US: Institute of Electrical and Electronics Engineers Inc..

MLA:

Li, Shiyu, et al. "GBOT: Graph-Based 3D Object Tracking for Augmented Reality-Assisted Assembly Guidance." Proceedings of the 31st IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2024, Orlando, FL Institute of Electrical and Electronics Engineers Inc., 2024. 513-523.

BibTeX: Download