Computational tissue staining of non-linear multimodal imaging using supervised and unsupervised deep learning

Pradhan P, Meyer T, Stallmach A, Waldner M, Schmitt M, Popp J, Bocklitz T, Vieth M (2021)

Publication Type: Journal article

Publication year: 2021

Journal

Biomedical Optics Express Optical Society of America

Book Volume: 12

Pages Range: 2280-2298

Journal Issue: 4

DOI: 10.1364/BOE.415962

Abstract

Hematoxylin and Eosin (H&E) staining is the 'gold-standard' method in histopathology. However, standard H&E staining of high-quality tissue sections requires long sample preparation times including sample embedding, which restricts its application for 'real-time' disease diagnosis. Due to this reason, a label-free alternative technique like non-linear multimodal (NLM) imaging, which is the combination of three non-linear optical modalities including coherent anti-Stokes Raman scattering, two-photon excitation fluorescence and second-harmonic generation, is proposed in this work. To correlate the information of the NLM images with H&E images, this work proposes computational staining of NLM images using deep learning models in a supervised and an unsupervised approach. In the supervised and the unsupervised approach, conditional generative adversarial networks (CGANs) and cycle conditional generative adversarial networks (cycle CGANs) are used, respectively. Both CGAN and cycle CGAN models generate pseudo H&E images, which are quantitatively analyzed based on mean squared error, structure similarity index and color shading similarity index. The mean of the three metrics calculated for the computationally generated H&E images indicate significant performance. Thus, utilizing CGAN and cycle CGAN models for computational staining is beneficial for diagnostic applications without performing a laboratory-based staining procedure. To the author's best knowledge, it is the first time that NLM images are computationally stained to H&E images using GANs in an unsupervised manner. Journal

Authors with CRIS profile

Maximilian Waldner Professur für Funktionelle Bildgebung in der Medizin Michael Vieth

Involved external institutions

Friedrich-Schiller-Universität Jena DE Germany (DE) Leibniz Health Technologies DE Germany (DE) Klinikum Bayreuth DE Germany (DE) Universitätsklinikum Jena DE Germany (DE)

How to cite

APA:

Pradhan, P., Meyer, T., Stallmach, A., Waldner, M., Schmitt, M., Popp, J.,... Vieth, M. (2021). Computational tissue staining of non-linear multimodal imaging using supervised and unsupervised deep learning. Biomedical Optics Express, 12(4), 2280-2298. https://doi.org/10.1364/BOE.415962

MLA:

Pradhan, Pranita, et al. "Computational tissue staining of non-linear multimodal imaging using supervised and unsupervised deep learning." Biomedical Optics Express 12.4 (2021): 2280-2298.

BibTeX: Download