DirectShape: Direct Photometric Alignment of Shape Priors for Visual Vehicle Pose and Shape Estimation

Wang R, Yang N, Stuckler J, Cremers D (2020)

Publication Type: Conference contribution

Publication year: 2020

Journal

Proceedings - IEEE International Conference on Robotics and Automation Institute of Electrical and Electronics Engineers

Publisher: Institute of Electrical and Electronics Engineers Inc.

Pages Range: 11067-11073

Conference Proceedings Title: Proceedings - IEEE International Conference on Robotics and Automation

Event location: Paris, FRA

ISBN: 9781728173955

DOI: 10.1109/ICRA40945.2020.9197095

Abstract

Scene understanding from images is a challenging problem encountered in autonomous driving. On the object level, while 2D methods have gradually evolved from computing simple bounding boxes to delivering finer grained results like instance segmentations, the 3D family is still dominated by estimating 3D bounding boxes. In this paper, we propose a novel approach to jointly infer the 3D rigid-body poses and shapes of vehicles from a stereo image pair using shape priors. Unlike previous works that geometrically align shapes to point clouds from dense stereo reconstruction, our approach works directly on images by combining a photometric and a silhouette alignment term in the energy function. An adaptive sparse point selection scheme is proposed to efficiently measure the consistency with both terms. In experiments, we show superior performance of our method on 3D pose and shape estimation over the previous geometric approach and demonstrate that our method can also be applied as a refinement step and significantly boost the performances of several state-of-the-art deep learning based 3D object detectors. All related materials and demonstration videos are available at the project page https://vision.in.tum.de/research/vslam/direct-shape.

Involved external institutions

Technische Universität München (TUM) DE Germany (DE) Max Planck Institute for Intelligent Systems (MPI-IS) / Max-Planck-Institut für Intelligente Systeme DE Germany (DE)

How to cite

APA:

Wang, R., Yang, N., Stuckler, J., & Cremers, D. (2020). DirectShape: Direct Photometric Alignment of Shape Priors for Visual Vehicle Pose and Shape Estimation. In Proceedings - IEEE International Conference on Robotics and Automation (pp. 11067-11073). Paris, FRA: Institute of Electrical and Electronics Engineers Inc..

MLA:

Wang, Rui, et al. "DirectShape: Direct Photometric Alignment of Shape Priors for Visual Vehicle Pose and Shape Estimation." Proceedings of the 2020 IEEE International Conference on Robotics and Automation, ICRA 2020, Paris, FRA Institute of Electrical and Electronics Engineers Inc., 2020. 11067-11073.

BibTeX: Download