Number of necessary training examples for Neural Networks with different number of trainable parameters

Götz TI, Göb S, Sawant S, Erick XF, Wittenberg T, Schmidkonz C, Tomé AM, Lang EW, Ramming A (2022)

Publication Type: Journal article

Publication year: 2022

Journal

Journal of Pathology Informatics Medknow Publications

Book Volume: 13

Article Number: 100114

DOI: 10.1016/j.jpi.2022.100114

Abstract

In this work, the network complexity should be reduced with a concomitant reduction in the number of necessary training examples. The focus thus was on the dependence of proper evaluation metrics on the number of adjustable parameters of the considered deep neural network. The used data set encompassed Hematoxylin and Eosin (H&E) colored cell images provided by various clinics. We used a deep convolutional neural network to get the relation between a model's complexity, its concomitant set of parameters, and the size of the training sample necessary to achieve a certain classification accuracy. The complexity of the deep neural networks was reduced by pruning a certain amount of filters in the network. As expected, the unpruned neural network showed best performance. The network with the highest number of trainable parameter achieved, within the estimated standard error of the optimized cross-entropy loss, best results up to 30% pruning. Strongly pruned networks are highly viable and the classification accuracy declines quickly with decreasing number of training patterns. However, up to a pruning ratio of 40%, we found a comparable performance of pruned and unpruned deep convolutional neural networks (DCNN) and densely connected convolutional networks (DCCN).

Authors with CRIS profile

Thomas Wittenberg
Christian Schmidkonz Lehrstuhl für Klinische Nuklearmedizin Andreas Ramming Professur für Immunologie der Organschädigung

Involved external institutions

Fraunhofer-Institut für Integrierte Schaltungen (IIS) DE Germany (DE) IEETA - Instituto de Engenharia Electrónica e Telemática de Aveiro PT Portugal (PT) Universität Regensburg DE Germany (DE)

How to cite

APA:

Götz, T.I., Göb, S., Sawant, S., Erick, X.F., Wittenberg, T., Schmidkonz, C.,... Ramming, A. (2022). Number of necessary training examples for Neural Networks with different number of trainable parameters. Journal of Pathology Informatics, 13. https://doi.org/10.1016/j.jpi.2022.100114

MLA:

Götz, Theresa Ida, et al. "Number of necessary training examples for Neural Networks with different number of trainable parameters." Journal of Pathology Informatics 13 (2022).

BibTeX: Download