3D-VField: Adversarial Augmentation of Point Clouds for Domain Generalization in 3D Object Detection

Lehner A, Gasperini S, Marcos-Ramiro A, Schmidt M, Mahani MAN, Navab N, Busam B, Tombari F (2022)

Publication Type: Conference contribution

Publication year: 2022

Journal

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition IEEE Computer Society; 1999

Publisher: IEEE Computer Society

Book Volume: 2022-June

Pages Range: 17274-17283

Conference Proceedings Title: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Event location: New Orleans, LA, USA

ISBN: 9781665469463

DOI: 10.1109/CVPR52688.2022.01678

Abstract

As 3D object detection on point clouds relies on the geometrical relationships between the points, non-standard object shapes can hinder a method's detection capability. However, in safety-critical settings, robustness to out-of-domain and long-tail samples is fundamental to circumvent dangerous issues, such as the misdetection of damaged or rare cars. In this work, we substantially improve the generalization of 3D object detectors to out-of-domain data by deforming point clouds during training. We achieve this with 3D-VField: a novel data augmentation method that plausibly deforms objects via vector fields learned in an adversarial fashion. Our approach constrains 3D points to slide along their sensor view rays while neither adding nor removing any of them. The obtained vectors are transferable, sample-independent and preserve shape and occlusions. Despite training only on a standard dataset, such as KITTI, augmenting with our vector fields significantly improves the generalization to differently shaped objects and scenes. Towards this end, we propose and share CrashD: a synthetic dataset of realistic damaged and rare cars, with a variety of crash scenarios. Extensive experiments on KITTI, Waymo, our CrashD and SUN RGB-D show the generalizability of our techniques to out-of-domain data, different models and sensors, namely LiDAR and ToF cameras, for both indoor and outdoor scenes. Our CrashD dataset is available at https://crashd-cars.github.io.

Involved external institutions

Technische Universität München (TUM) DE Germany (DE) BMW AG - Bayerische Motoren Werke / BMW Group DE Germany (DE)

How to cite

APA:

Lehner, A., Gasperini, S., Marcos-Ramiro, A., Schmidt, M., Mahani, M.-A.N., Navab, N.,... Tombari, F. (2022). 3D-VField: Adversarial Augmentation of Point Clouds for Domain Generalization in 3D Object Detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 17274-17283). New Orleans, LA, USA: IEEE Computer Society.

MLA:

Lehner, Alexander, et al. "3D-VField: Adversarial Augmentation of Point Clouds for Domain Generalization in 3D Object Detection." Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA IEEE Computer Society, 2022. 17274-17283.

BibTeX: Download