Effective Version Space Reduction for Convolutional Neural Networks
Liu J, Chiotellis I, Triebel R, Cremers D (2021)
Publication Type: Conference contribution
Publication year: 2021
Journal
Publisher: Springer Science and Business Media Deutschland GmbH
Book Volume: 12458 LNAI
Pages Range: 85-100
Conference Proceedings Title: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Event location: Virtual, Online
ISBN: 9783030676605
DOI: 10.1007/978-3-030-67661-2_6
Abstract
In active learning, sampling bias could pose a serious inconsistency problem and hinder the algorithm from finding the optimal hypothesis. However, many methods for neural networks are hypothesis space agnostic and do not address this problem. We examine active learning with convolutional neural networks through the principled lens of version space reduction. We identify the connection between two approaches – prior mass reduction and diameter reduction – and propose a new diameter-based querying method – the minimum Gibbs-vote disagreement. By estimating version space diameter and bias, we illustrate how version space of neural networks evolves and examine the realizability assumption. With experiments on MNIST, Fashion-MNIST, SVHN and STL-10 datasets, we demonstrate that diameter reduction methods reduce the version space more effectively and perform better than prior mass reduction and other baselines, and that the Gibbs vote disagreement is on par with the best query method.
Involved external institutions
Germany (DE)How to cite
APA:
Liu, J., Chiotellis, I., Triebel, R., & Cremers, D. (2021). Effective Version Space Reduction for Convolutional Neural Networks. In Frank Hutter, Kristian Kersting, Jefrey Lijffijt, Isabel Valera (Eds.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 85-100). Virtual, Online: Springer Science and Business Media Deutschland GmbH.
MLA:
Liu, Jiayu, et al. "Effective Version Space Reduction for Convolutional Neural Networks." Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2020, Virtual, Online Ed. Frank Hutter, Kristian Kersting, Jefrey Lijffijt, Isabel Valera, Springer Science and Business Media Deutschland GmbH, 2021. 85-100.
BibTeX: Download