SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences
Wu SC, Wald J, Tateno K, Navab N, Tombari F (2021)
Publication Type: Conference contribution
Publication year: 2021
Journal
Publisher: IEEE Computer Society
Pages Range: 7511-7521
Conference Proceedings Title: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Event location: Virtual, Online, USA
ISBN: 9781665445092
DOI: 10.1109/CVPR46437.2021.00743
Abstract
Scene graphs are a compact and explicit representation successfully used in a variety of 2D scene understanding tasks. This work proposes a method to incrementally build up semantic scene graphs from a 3D environment given a sequence of RGB-D frames. To this end, we aggregate PointNet features from primitive scene components by means of a graph neural network. We also propose a novel attention mechanism well suited for partial and missing graph data present in such an incremental reconstruction scenario. Although our proposed method is designed to run on submaps of the scene, we show it also transfers to entire 3D scenes. Experiments show that our approach outperforms 3D scene graph prediction methods by a large margin and its accuracy is on par with other 3D semantic and panoptic segmentation methods while running at 35Hz.
Involved external institutions
Germany (DE)
United States (USA) (US)How to cite
APA:
Wu, S.-C., Wald, J., Tateno, K., Navab, N., & Tombari, F. (2021). SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 7511-7521). Virtual, Online, USA: IEEE Computer Society.
MLA:
Wu, Shun-Cheng, et al. "SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences." Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2021, Virtual, Online, USA IEEE Computer Society, 2021. 7511-7521.
BibTeX: Download