A Unified Framework for Implicit Sinkhorn Differentiation

Eisenberger M, Toker A, Leal-Taixe L, Bernard F, Cremers D (2022)

Publication Type: Conference contribution

Publication year: 2022

Journal

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition IEEE Computer Society; 1999

Publisher: IEEE Computer Society

Book Volume: 2022-June

Pages Range: 499-508

Conference Proceedings Title: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Event location: New Orleans, LA, USA

ISBN: 9781665469463

DOI: 10.1109/CVPR52688.2022.00059

Abstract

The Sinkhorn operator has recently experienced a surge of popularity in computer vision and related fields. One major reason is its ease of integration into deep learning frameworks. To allow for an efficient training of respective neural networks, we propose an algorithm that obtains analytical gradients of a Sinkhorn layer via implicit differentiation. In comparison to prior work, our framework is based on the most general formulation of the Sinkhorn operator. It allows for any type of loss function, while both the target capacities and cost matrices are differentiated jointly. We further construct error bounds of the resulting algorithm for approximate inputs. Finally, we demonstrate that for a number of applications, simply replacing automatic differentiation with our algorithm directly improves the stability and accuracy of the obtained gradients. Moreover, we show that it is computationally more efficient, particularly when resources like GPU memory are scarce. 11Our implementation is available under the following link: https://github.com/marvin-eisenberger/implicit-sinkhorn

Involved external institutions

Technische Universität München (TUM) DE Germany (DE) Rheinische Friedrich-Wilhelms-Universität Bonn DE Germany (DE)

How to cite

APA:

Eisenberger, M., Toker, A., Leal-Taixe, L., Bernard, F., & Cremers, D. (2022). A Unified Framework for Implicit Sinkhorn Differentiation. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 499-508). New Orleans, LA, USA: IEEE Computer Society.

MLA:

Eisenberger, Marvin, et al. "A Unified Framework for Implicit Sinkhorn Differentiation." Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA IEEE Computer Society, 2022. 499-508.

BibTeX: Download