ForkNet: Multi-branch volumetric semantic completion from a single depth image
Wang Y, Tan DJ, Navab N, Tombari F (2019)
Publication Type: Conference contribution
Publication year: 2019
Publisher: Institute of Electrical and Electronics Engineers Inc.
Book Volume: 2019-October
Pages Range: 8607-8616
Conference Proceedings Title: Proceedings of the IEEE International Conference on Computer Vision
Event location: Seoul, KOR
ISBN: 9781728148038
Abstract
We propose a novel model for 3D semantic completion from a single depth image, based on a single encoder and three separate generators used to reconstruct different geometric and semantic representations of the original and completed scene, all sharing the same latent space. To transfer information between the geometric and semantic branches of the network, we introduce paths between them concatenating features at corresponding network layers. Motivated by the limited amount of training samples from real scenes, an interesting attribute of our architecture is the capacity to supplement the existing dataset by generating a new training dataset with high quality, realistic scenes that even includes occlusion and real noise. We build the new dataset by sampling the features directly from latent space which generates a pair of partial volumetric surface and completed volumetric semantic surface. Moreover, we utilize multiple discriminators to increase the accuracy and realism of the reconstructions. We demonstrate the benefits of our approach on standard benchmarks for the two most common completion tasks: Semantic 3D scene completion and 3D object completion.
Involved external institutions
Germany (DE)
United States (USA) (US)How to cite
APA:
Wang, Y., Tan, D.J., Navab, N., & Tombari, F. (2019). ForkNet: Multi-branch volumetric semantic completion from a single depth image. In Proceedings of the IEEE International Conference on Computer Vision (pp. 8607-8616). Seoul, KOR: Institute of Electrical and Electronics Engineers Inc..
MLA:
Wang, Yida, et al. "ForkNet: Multi-branch volumetric semantic completion from a single depth image." Proceedings of the 17th IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, KOR Institute of Electrical and Electronics Engineers Inc., 2019. 8607-8616.
BibTeX: Download