MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera
Wimbauer F, Yang N, Von Stumberg L, Zeller N, Cremers D (2021)
Publication Type: Conference contribution
Publication year: 2021
Journal
Publisher: IEEE Computer Society
Pages Range: 6108-6118
Conference Proceedings Title: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Event location: Virtual, Online, USA
ISBN: 9781665445092
DOI: 10.1109/CVPR46437.2021.00605
Abstract
In this paper, we propose MonoRec, a semi-supervised monocular dense reconstruction architecture that predicts depth maps from a single moving camera in dynamic environments. MonoRec is based on a multi-view stereo setting which encodes the information of multiple consecutive images in a cost volume. To deal with dynamic objects in the scene, we introduce a MaskModule that predicts moving object masks by leveraging the photometric inconsistencies encoded in the cost volumes. Unlike other multi-view stereo methods, MonoRec is able to reconstruct both static and moving objects by leveraging the predicted masks. Furthermore, we present a novel multi-stage training scheme with a semi-supervised loss formulation that does not require LiDAR depth values. We carefully evaluate MonoRec on the KITTI dataset and show that it achieves state-of-the-art performance compared to both multi-view and single-view methods. With the model trained on KITTI, we furthermore demonstrate that MonoRec is able to generalize well to both the Oxford RobotCar dataset and the more challenging TUM-Mono dataset recorded by a handheld camera. Code and related materials are available at https://vision.in.tum.de/research/monorec.
Involved external institutions
Germany (DE)How to cite
APA:
Wimbauer, F., Yang, N., Von Stumberg, L., Zeller, N., & Cremers, D. (2021). MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 6108-6118). Virtual, Online, USA: IEEE Computer Society.
MLA:
Wimbauer, Felix, et al. "MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera." Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2021, Virtual, Online, USA IEEE Computer Society, 2021. 6108-6118.
BibTeX: Download