End-to-End Lidar-Camera Self-Calibration for Autonomous Vehicles

Rachman A, Seiler J, Kaup A (2023)

Publication Type: Conference contribution, Conference Contribution

Publication year: 2023

Publisher: Institute of Electrical and Electronics Engineers Inc.

Book Volume: 2023-June

Pages Range: 1-6

Conference Proceedings Title: IEEE Intelligent Vehicles Symposium, Proceedings

Event location: Anchorage US

ISBN: 9798350346916

DOI: 10.1109/IV55152.2023.10186613

Abstract

Autonomous vehicles are equipped with a multi-modal sensor setup to enable the car to drive safely. The initial calibration of such perception sensors is a highly matured topic and is routinely done in an automated factory environment. However, an intriguing question arises on how to maintain the calibration quality throughout the vehicle's operating duration. Another challenge is to calibrate multiple sensors jointly to ensure no propagation of systemic errors. In this paper, we propose Camera Lidar Calibration Network (CaLiCaNet), an end-to-end deep self-calibration network which addresses the automatic calibration problem for pinhole camera and Lidar. We jointly predict the camera intrinsic parameters (focal length and distortion) as well as Lidar-Camera extrinsic parameters (rotation and translation), by regressing feature correlation between the camera image and the Lidar point cloud. The network is arranged in a Siamese-twin structure to constrain the network features learning to a mutually shared feature in both point cloud and camera (Lidar-camera constraint). Evaluation using KITTI datasets shows that we achieve 0.154° and 0.059 m accuracy with a reprojection error of 0.028 pixel with a single-pass inference. We also provide an ablative study of how our end-to-end learning architecture offers lower terminal loss (21% decrease in rotation loss) compared to isolated calibration.

Authors with CRIS profile

Arya Rachman Lehrstuhl für Multimediakommunikation und Signalverarbeitung (LMS) Jürgen Seiler Lehrstuhl für Multimediakommunikation und Signalverarbeitung (LMS) Andre Kaup Lehrstuhl für Multimediakommunikation und Signalverarbeitung (LMS)

How to cite

APA:

Rachman, A., Seiler, J., & Kaup, A. (2023). End-to-End Lidar-Camera Self-Calibration for Autonomous Vehicles. In IEEE Intelligent Vehicles Symposium, Proceedings (pp. 1-6). Anchorage, US: Institute of Electrical and Electronics Engineers Inc..

MLA:

Rachman, Arya, Jürgen Seiler, and André Kaup. "End-to-End Lidar-Camera Self-Calibration for Autonomous Vehicles." Proceedings of the IEEE Intelligent Vehicles Symposium, Anchorage Institute of Electrical and Electronics Engineers Inc., 2023. 1-6.

BibTeX: Download