B-Cos Aligned Transformers Learn Human-Interpretable Features
Tran M, Lahiani A, Dicente Cid Y, Boxberg M, Lienemann P, Matek C, Wagner SJ, Theis FJ, Klaiman E, Peng T (2023)
Publication Type: Conference contribution
Publication year: 2023
Journal
Publisher: Springer Science and Business Media Deutschland GmbH
Book Volume: 14227 LNCS
Pages Range: 514-524
Conference Proceedings Title: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Event location: Vancouver, BC, CAN
ISBN: 9783031439926
DOI: 10.1007/978-3-031-43993-3_50
Abstract
Vision Transformers (ViTs) and Swin Transformers (Swin) are currently state-of-the-art in computational pathology. However, domain experts are still reluctant to use these models due to their lack of interpretability. This is not surprising, as critical decisions need to be transparent and understandable. The most common approach to understanding transformers is to visualize their attention. However, attention maps of ViTs are often fragmented, leading to unsatisfactory explanations. Here, we introduce a novel architecture called the B-cos Vision Transformer (BvT) that is designed to be more interpretable. It replaces all linear transformations with the B-cos transform to promote weight-input alignment. In a blinded study, medical experts clearly ranked BvTs above ViTs, suggesting that our network is better at capturing biomedically relevant structures. This is also true for the B-cos Swin Transformer (Bwin). Compared to the Swin Transformer, it even improves the F1-score by up to 4.7% on two public datasets.
Involved external institutions
Roche Diagnostics GmbH
Germany (DE)
Technische Universität München (TUM)
Germany (DE)
Helmholtz-Gemeinschaft / Hermann von Helmholtz-Gemeinschaft Deutscher Forschungszentren e.V.
Germany (DE)
Germany (DE)
Technische Universität München (TUM)
Germany (DE)
Helmholtz-Gemeinschaft / Hermann von Helmholtz-Gemeinschaft Deutscher Forschungszentren e.V.
Germany (DE)
How to cite
APA:
Tran, M., Lahiani, A., Dicente Cid, Y., Boxberg, M., Lienemann, P., Matek, C.,... Peng, T. (2023). B-Cos Aligned Transformers Learn Human-Interpretable Features. In Hayit Greenspan, Hayit Greenspan, Anant Madabhushi, Parvin Mousavi, Septimiu Salcudean, James Duncan, Tanveer Syeda-Mahmood, Russell Taylor (Eds.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 514-524). Vancouver, BC, CAN: Springer Science and Business Media Deutschland GmbH.
MLA:
Tran, Manuel, et al. "B-Cos Aligned Transformers Learn Human-Interpretable Features." Proceedings of the 26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023, Vancouver, BC, CAN Ed. Hayit Greenspan, Hayit Greenspan, Anant Madabhushi, Parvin Mousavi, Septimiu Salcudean, James Duncan, Tanveer Syeda-Mahmood, Russell Taylor, Springer Science and Business Media Deutschland GmbH, 2023. 514-524.
BibTeX: Download