Attention meets Geometry: Geometry Guided Spatial-Temporal Attention for Consistent Self-Supervised Monocular Depth Estimation
Ruhkamp P, Gao D, Chen H, Navab N, Busam B (2021)
Publication Type: Conference contribution
Publication year: 2021
Publisher: Institute of Electrical and Electronics Engineers Inc.
Pages Range: 837-847
Conference Proceedings Title: Proceedings - 2021 International Conference on 3D Vision, 3DV 2021
Event location: Virtual, Online, GBR
ISBN: 9781665426886
DOI: 10.1109/3DV53792.2021.00092
Abstract
Inferring geometrically consistent dense 3D scenes across a tuple of temporally consecutive images remains challenging for self-supervised monocular depth prediction pipelines. This paper explores how the increasingly popular transformer architecture,together with novel regularized loss formulations,can improve depth consistency while preserving accuracy. We propose a spatial attention module that correlates coarse depth predictions to aggregate local geometric information. A novel temporal attention mechanism further processes the local geometric information in a global context across consecutive images. Additionally,we introduce geometric constraints between frames regularized by photometric cycle consistency. By combining our proposed regularization and the novel spatial-temporal-attention module we fully leverage both the geometric and appearance-based consistency across monocular frames. This yields geometrically meaningful attention and improves temporal depth stability and accuracy compared to previous methods.
Involved external institutions
Germany (DE)How to cite
APA:
Ruhkamp, P., Gao, D., Chen, H., Navab, N., & Busam, B. (2021). Attention meets Geometry: Geometry Guided Spatial-Temporal Attention for Consistent Self-Supervised Monocular Depth Estimation. In Proceedings - 2021 International Conference on 3D Vision, 3DV 2021 (pp. 837-847). Virtual, Online, GBR: Institute of Electrical and Electronics Engineers Inc..
MLA:
Ruhkamp, Patrick, et al. "Attention meets Geometry: Geometry Guided Spatial-Temporal Attention for Consistent Self-Supervised Monocular Depth Estimation." Proceedings of the 9th International Conference on 3D Vision, 3DV 2021, Virtual, Online, GBR Institute of Electrical and Electronics Engineers Inc., 2021. 837-847.
BibTeX: Download