R4Dyn: Exploring Radar for Self-Supervised Monocular Depth Estimation of Dynamic Scenes

Gasperini S, Koch P, Dallabetta V, Navab N, Busam B, Tombari F (2021)

Publication Type: Conference contribution

Publication year: 2021

Publisher: Institute of Electrical and Electronics Engineers Inc.

Pages Range: 751-760

Conference Proceedings Title: Proceedings - 2021 International Conference on 3D Vision, 3DV 2021

Event location: Virtual, Online, GBR

ISBN: 9781665426886

DOI: 10.1109/3DV53792.2021.00084

Abstract

While self-supervised monocular depth estimation in driving scenarios has achieved comparable performance to supervised approaches,violations of the static world assumption can still lead to erroneous depth predictions of traffic participants,posing a potential safety issue. In this paper,we present R4Dyn,a novel set of techniques to use cost-efficient radar data on top of a self-supervised depth estimation framework. In particular,we show how radar can be used during training as weak supervision signal,as well as an extra input to enhance the estimation robustness at inference time. Since automotive radars are readily available,this allows to collect training data from a variety of existing vehicles. Moreover,by filtering and expanding the signal to make it compatible with learning-based approaches,we address radar inherent issues,such as noise and sparsity. With R4Dyn we are able to overcome a major limitation of self-supervised depth estimation,i.e. the prediction of traffic participants. We substantially improve the estimation on dynamic objects,such as cars by 37% on the challenging nuScenes dataset,hence demonstrating that radar is a valuable additional sensor for monocular depth estimation in autonomous vehicles.

Involved external institutions

Technische Universität München (TUM) DE Germany (DE) BMW AG - Bayerische Motoren Werke / BMW Group DE Germany (DE)

How to cite

APA:

Gasperini, S., Koch, P., Dallabetta, V., Navab, N., Busam, B., & Tombari, F. (2021). R4Dyn: Exploring Radar for Self-Supervised Monocular Depth Estimation of Dynamic Scenes. In Proceedings - 2021 International Conference on 3D Vision, 3DV 2021 (pp. 751-760). Virtual, Online, GBR: Institute of Electrical and Electronics Engineers Inc..

MLA:

Gasperini, Stefano, et al. "R4Dyn: Exploring Radar for Self-Supervised Monocular Depth Estimation of Dynamic Scenes." Proceedings of the 9th International Conference on 3D Vision, 3DV 2021, Virtual, Online, GBR Institute of Electrical and Electronics Engineers Inc., 2021. 751-760.

BibTeX: Download