Fine-Grained Neural Network Explanation by Identifying Input Features with Predictive Information

Zhang Y, Khakzar A, Li Y, Farshad A, Kim ST, Navab N (2021)

Publication Type: Conference contribution

Publication year: 2021

Publisher: Neural information processing systems foundation

Book Volume: 24

Pages Range: 20040-20051

Conference Proceedings Title: Advances in Neural Information Processing Systems

Event location: Virtual, Online

ISBN: 9781713845393

Abstract

One principal approach for illuminating a black-box neural network is feature attribution, i.e. identifying the importance of input features for the network's prediction. The predictive information of features is recently proposed as a proxy for the measure of their importance. So far, the predictive information is only identified for latent features by placing an information bottleneck within the network. We propose a method to identify features with predictive information in the input domain. The method results in fine-grained identification of input features' information and is agnostic to network architecture. The core idea of our method is leveraging a bottleneck on the input that only lets input features associated with predictive latent features pass through. We compare our method with several feature attribution methods using mainstream feature attribution evaluation experiments. The code 1 is publicly available.

Involved external institutions

Technische Universität München (TUM) DE Germany (DE) Kyung Hee University KR Korea, Republic of (KR) Klinikum der Universität München DE Germany (DE)

How to cite

APA:

Zhang, Y., Khakzar, A., Li, Y., Farshad, A., Kim, S.T., & Navab, N. (2021). Fine-Grained Neural Network Explanation by Identifying Input Features with Predictive Information. In Marc'Aurelio Ranzato, Alina Beygelzimer, Yann Dauphin, Percy S. Liang, Jenn Wortman Vaughan (Eds.), Advances in Neural Information Processing Systems (pp. 20040-20051). Virtual, Online: Neural information processing systems foundation.

MLA:

Zhang, Yang, et al. "Fine-Grained Neural Network Explanation by Identifying Input Features with Predictive Information." Proceedings of the 35th Conference on Neural Information Processing Systems, NeurIPS 2021, Virtual, Online Ed. Marc'Aurelio Ranzato, Alina Beygelzimer, Yann Dauphin, Percy S. Liang, Jenn Wortman Vaughan, Neural information processing systems foundation, 2021. 20040-20051.

BibTeX: Download