Post-hoc Uncertainty Calibration for Domain Drift Scenarios

Tomani C, Gruber S, Erdem ME, Cremers D, Buettner F (2021)

Publication Type: Conference contribution

Publication year: 2021

Journal

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition IEEE Computer Society; 1999

Publisher: IEEE Computer Society

Pages Range: 10119-10127

Conference Proceedings Title: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Event location: Virtual, Online, USA

ISBN: 9781665445092

DOI: 10.1109/CVPR46437.2021.00999

Abstract

We address the problem of uncertainty calibration. While standard deep neural networks typically yield uncalibrated predictions, calibrated confidence scores that are representative of the true likelihood of a prediction can be achieved using post-hoc calibration methods. However, to date, the focus of these approaches has been on in-domain calibration. Our contribution is two-fold. First, we show that existing post-hoc calibration methods yield highly overconfident predictions under domain shift. Second, we introduce a simple strategy where perturbations are applied to samples in the validation set before performing the post-hoc calibration step. In extensive experiments, we demonstrate that this perturbation step results in substantially better calibration under domain shift on a wide range of architectures and modelling tasks.

Involved external institutions

Technische Universität München (TUM) DE Germany (DE) Deutsches Krebsforschungszentrum (DKFZ) DE Germany (DE)

How to cite

APA:

Tomani, C., Gruber, S., Erdem, M.E., Cremers, D., & Buettner, F. (2021). Post-hoc Uncertainty Calibration for Domain Drift Scenarios. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 10119-10127). Virtual, Online, USA: IEEE Computer Society.

MLA:

Tomani, Christian, et al. "Post-hoc Uncertainty Calibration for Domain Drift Scenarios." Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2021, Virtual, Online, USA IEEE Computer Society, 2021. 10119-10127.

BibTeX: Download