FuseNet: Incorporating depth into semantic segmentation via fusion-based CNN architecture
Hazirbas C, Ma L, Domokos C, Cremers D (2017)
Publication Type: Conference contribution
Publication year: 2017
Journal
Publisher: Springer Verlag
Book Volume: 10111 LNCS
Conference Proceedings Title: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Event location: Taipei, TWN
ISBN: 9783319541808
DOI: 10.1007/978-3-319-54181-5_14
Abstract
In this paper we address the problem of semantic labeling of indoor scenes on RGB-D data. With the availability of RGB-D cameras, it is expected that additional depth measurement will improve the accuracy. Here we investigate a solution how to incorporate complementary depth information into a semantic segmentation framework by making use of convolutional neural networks (CNNs). Recently encoder-decoder type fully convolutional CNN architectures have achieved a great success in the field of semantic segmentation. Motivated by this observation we propose an encoder-decoder type network, where the encoder part is composed of two branches of networks that simultaneously extract features from RGB and depth images and fuse depth features into the RGB feature maps as the network goes deeper. Comprehensive experimental evaluations demonstrate that the proposed fusion-based architecture achieves competitive results with the state-of-the-art methods on the challenging SUN RGB-D benchmark obtaining 76.27% global accuracy, 48.30% average class accuracy and 37.29% average intersection-over-union score.
Involved external institutions
Germany (DE)How to cite
APA:
Hazirbas, C., Ma, L., Domokos, C., & Cremers, D. (2017). FuseNet: Incorporating depth into semantic segmentation via fusion-based CNN architecture. In Yoichi Sato, Ko Nishino, Vincent Lepetit, Shang-Hong Lai (Eds.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Taipei, TWN: Springer Verlag.
MLA:
Hazirbas, Caner, et al. "FuseNet: Incorporating depth into semantic segmentation via fusion-based CNN architecture." Proceedings of the 13th Asian Conference on Computer Vision, ACCV 2016, Taipei, TWN Ed. Yoichi Sato, Ko Nishino, Vincent Lepetit, Shang-Hong Lai, Springer Verlag, 2017.
BibTeX: Download