What Can We Learn About a Generated Image Corrupting Its Latent Representation?

Tomczak A, Gupta A, Ilic S, Navab N, Albarqouni S (2022)

Publication Type: Conference contribution

Publication year: 2022

Journal

Lecture Notes in Computer Science Springer Verlag

Publisher: Springer Science and Business Media Deutschland GmbH

Book Volume: 13436 LNCS

Pages Range: 505-515

Conference Proceedings Title: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Event location: Singapore, SGP

ISBN: 9783031164453

DOI: 10.1007/978-3-031-16446-0_48

Abstract

Generative adversarial networks (GANs) offer an effective solution to the image-to-image translation problem, thereby allowing for new possibilities in medical imaging. They can translate images from one imaging modality to another at a low cost. For unpaired datasets, they rely mostly on cycle loss. Despite its effectiveness in learning the underlying data distribution, it can lead to a discrepancy between input and output data. The purpose of this work is to investigate the hypothesis that we can predict image quality based on its latent representation in the GANs bottleneck. We achieve this by corrupting the latent representation with noise and generating multiple outputs. The degree of differences between them is interpreted as the strength of the representation: the more robust the latent representation, the fewer changes in the output image the corruption causes. Our results demonstrate that our proposed method has the ability to i) predict uncertain parts of synthesized images, and ii) identify samples that may not be reliable for downstream tasks, e.g., liver segmentation task.

Involved external institutions

Technische Universität München (TUM) DE Germany (DE) Indian Institute of Technology (IIT), Delhi IN India (IN)

How to cite

APA:

Tomczak, A., Gupta, A., Ilic, S., Navab, N., & Albarqouni, S. (2022). What Can We Learn About a Generated Image Corrupting Its Latent Representation? In Linwei Wang, Qi Dou, P. Thomas Fletcher, Stefanie Speidel, Shuo Li (Eds.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 505-515). Singapore, SGP: Springer Science and Business Media Deutschland GmbH.

MLA:

Tomczak, Agnieszka, et al. "What Can We Learn About a Generated Image Corrupting Its Latent Representation?" Proceedings of the 25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022, Singapore, SGP Ed. Linwei Wang, Qi Dou, P. Thomas Fletcher, Stefanie Speidel, Shuo Li, Springer Science and Business Media Deutschland GmbH, 2022. 505-515.

BibTeX: Download