Radiological Reports Improve Pre-training for Localized Imaging Tasks on Chest X-Rays
Mueller P, Kaissis G, Zou C, Rueckert D (2022)
Publication Type: Conference contribution
Publication year: 2022
Journal
Publisher: Springer Science and Business Media Deutschland GmbH
Book Volume: 13435 LNCS
Pages Range: 647-657
Conference Proceedings Title: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Event location: Singapore, SGP
ISBN: 9783031164422
DOI: 10.1007/978-3-031-16443-9_62
Abstract
Self-supervised pre-training on unlabeled images has shown promising results in the medical domain. Recently, methods using text-supervision from companion text like radiological reports improved upon these results even further. However, most works in the medical domain focus on image classification downstream tasks and do not study more localized tasks like semantic segmentation or object detection. We therefore propose a novel evaluation framework consisting of 18 localized tasks, including semantic segmentation and object detection, on five public chest radiography datasets. Using our proposed evaluation framework, we study the effectiveness of existing text-supervised methods and compare them with image-only self-supervised methods and transfer from classification in more than 1200 evaluation runs. Our experiments show that text-supervised methods outperform all other methods on 13 out of 18 tasks making them the preferred method. In conclusion, image-only contrastive methods provide a strong baseline if no reports are available while transfer from classification, even in-domain, does not perform well in pre-training for localized tasks.
Involved external institutions
Germany (DE)How to cite
APA:
Mueller, P., Kaissis, G., Zou, C., & Rueckert, D. (2022). Radiological Reports Improve Pre-training for Localized Imaging Tasks on Chest X-Rays. In Linwei Wang, Qi Dou, P. Thomas Fletcher, Stefanie Speidel, Shuo Li (Eds.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 647-657). Singapore, SGP: Springer Science and Business Media Deutschland GmbH.
MLA:
Mueller, Philip, et al. "Radiological Reports Improve Pre-training for Localized Imaging Tasks on Chest X-Rays." Proceedings of the 25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022, Singapore, SGP Ed. Linwei Wang, Qi Dou, P. Thomas Fletcher, Stefanie Speidel, Shuo Li, Springer Science and Business Media Deutschland GmbH, 2022. 647-657.
BibTeX: Download