Highly accurate classification of chest radiographic reports using a deep learning natural language model pre-trained on 3.8 million text reports
Bressem KK, Adams LC, Gaudin RA, Troeltzsch D, Hamm B, Makowski MR, Schuele CY, Vahldiek JL, Niehues SM (2020)
Publication Type: Journal article
Publication year: 2020
Journal
Book Volume: 36
Pages Range: 5255-5261
Journal Issue: 21
DOI: 10.1093/bioinformatics/btaa668
Abstract
Motivation: The development of deep, bidirectional transformers such as Bidirectional Encoder Representations from Transformers (BERT) led to an outperformance of several Natural Language Processing (NLP) benchmarks. Especially in radiology, large amounts of free-text data are generated in daily clinical workflow. These report texts could be of particular use for the generation of labels in machine learning, especially for image classification. However, as report texts are mostly unstructured, advanced NLP methods are needed to enable accurate text classification. While neural networks can be used for this purpose, they must first be trained on large amounts of manually labelled data to achieve good results. In contrast, BERT models can be pre-trained on unlabelled data and then only require fine tuning on a small amount of manually labelled data to achieve even better results. Results: Using BERT to identify the most important findings in intensive care chest radiograph reports, we achieve areas under the receiver operation characteristics curve of 0.98 for congestion, 0.97 for effusion, 0.97 for consolidation and 0.99 for pneumothorax, surpassing the accuracy of previous approaches with comparatively little annotation effort. Our approach could therefore help to improve information extraction from free-text medical reports. Availability and implementation We make the source code for fine-tuning the BERT-models freely available at https://github.com/fast-raidiology/bert-for-radiology.
Involved external institutions
Germany (DE)How to cite
APA:
Bressem, K.K., Adams, L.C., Gaudin, R.A., Troeltzsch, D., Hamm, B., Makowski, M.R.,... Niehues, S.M. (2020). Highly accurate classification of chest radiographic reports using a deep learning natural language model pre-trained on 3.8 million text reports. Bioinformatics, 36(21), 5255-5261. https://doi.org/10.1093/bioinformatics/btaa668
MLA:
Bressem, Keno K., et al. "Highly accurate classification of chest radiographic reports using a deep learning natural language model pre-trained on 3.8 million text reports." Bioinformatics 36.21 (2020): 5255-5261.
BibTeX: Download