Home Publications Research Grants Inventions & Patents Awards Additional Research Activities Faculties & Institutions Research Areas

Neural response interpretation through the lens of critical pathways

Khakzar A, Baselizadeh S, Khanduja S, Rupprecht C, Kim ST, Navab N (2021)

---

Publication Type: Conference contribution

Publication year: 2021

Journal

Publisher: IEEE Computer Society

Pages Range: 13523-13533

Conference Proceedings Title: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Event location: Virtual, Online, USA

ISBN: 9781665445092

DOI: 10.1109/CVPR46437.2021.01332

Abstract

Is critical input information encoded in specific sparse pathways within the neural network? In this work, we discuss the problem of identifying these critical pathways and subsequently leverage them for interpreting the network's response to an input. The pruning objective - selecting the smallest group of neurons for which the response remains equivalent to the original network - has been previously proposed for identifying critical pathways. We demonstrate that sparse pathways derived from pruning do not necessarily encode critical input information. To ensure sparse pathways include critical fragments of the encoded input information, we propose pathway selection via neurons' contribution to the response. We proceed to explain how critical pathways can reveal critical input features. We prove that pathways selected via neuron contribution are locally linear (in an `2-ball), a property that we use for proposing a feature attribution method: “pathway gradient”. We validate our interpretation method using mainstream evaluation experiments. The validation of pathway gradient interpretation method further confirms that selected pathways using neuron contributions correspond to critical input features. The code^{1 2} is publicly available.

Involved external institutions

How to cite

APA:

Khakzar, A., Baselizadeh, S., Khanduja, S., Rupprecht, C., Kim, S.T., & Navab, N. (2021). Neural response interpretation through the lens of critical pathways. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 13523-13533). Virtual, Online, USA: IEEE Computer Society.

MLA:

Khakzar, Ashkan, et al. "Neural response interpretation through the lens of critical pathways." Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2021, Virtual, Online, USA IEEE Computer Society, 2021. 13523-13533.

BibTeX: Download