Deep virtual stereo odometry: Leveraging deep depth prediction for monocular direct sparse odometry
Yang N, Wang R, Stueckler J, Cremers D (2018)
Publication Type: Conference contribution
Publication year: 2018
Journal
Publisher: Springer Verlag
Book Volume: 11212 LNCS
Pages Range: 835-852
Conference Proceedings Title: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Event location: Munich, DEU
ISBN: 9783030012366
DOI: 10.1007/978-3-030-01237-3_50
Abstract
Monocular visual odometry approaches that purely rely on geometric cues are prone to scale drift and require sufficient motion parallax in successive frames for motion estimation and 3D reconstruction. In this paper, we propose to leverage deep monocular depth prediction to overcome limitations of geometry-based monocular visual odometry. To this end, we incorporate deep depth predictions into Direct Sparse Odometry (DSO) as direct virtual stereo measurements. For depth prediction, we design a novel deep network that refines predicted depth from a single image in a two-stage process. We train our network in a semi-supervised way on photoconsistency in stereo images and on consistency with accurate sparse depth reconstructions from Stereo DSO. Our deep predictions excel state-of-the-art approaches for monocular depth on the KITTI benchmark. Moreover, our Deep Virtual Stereo Odometry clearly exceeds previous monocular and deep-learning based methods in accuracy. It even achieves comparable performance to the state-of-the-art stereo methods, while only relying on a single camera.
Involved external institutions
Germany (DE)How to cite
APA:
Yang, N., Wang, R., Stueckler, J., & Cremers, D. (2018). Deep virtual stereo odometry: Leveraging deep depth prediction for monocular direct sparse odometry. In Vittorio Ferrari, Cristian Sminchisescu, Yair Weiss, Martial Hebert (Eds.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 835-852). Munich, DEU: Springer Verlag.
MLA:
Yang, Nan, et al. "Deep virtual stereo odometry: Leveraging deep depth prediction for monocular direct sparse odometry." Proceedings of the 15th European Conference on Computer Vision, ECCV 2018, Munich, DEU Ed. Vittorio Ferrari, Cristian Sminchisescu, Yair Weiss, Martial Hebert, Springer Verlag, 2018. 835-852.
BibTeX: Download