Nikolaidou K, Retsinas G, Christlein V, Seuret M, Sfikas G, Barney Smith E, Mokayed H, Liwicki M (2023)
Publication Language: English
Publication Type: Conference contribution
Publication year: 2023
Publisher: Springer
City/Town: Cham
Book Volume: 14188
Pages Range: 384-401
Conference Proceedings Title: Lecture Notes in Computer Science
ISBN: 9783031416781
DOI: 10.1007/978-3-031-41679-8_22
Open Access Link: https://arxiv.org/abs/2303.16576
Text-to-Image synthesis is the task of generating an image according to a specific text description. Generative Adversarial Networks have been considered the standard method for image synthesis virtually since their introduction. Denoising Diffusion Probabilistic Models are recently setting a new baseline, with remarkable results in Text-to-Image synthesis, among other fields. Aside its usefulness per se, it can also be particularly relevant as a tool for data augmentation to aid training models for other document image processing tasks. In this work, we present a latent diffusion-based method for styled text-to-text-content-image generation on word-level. Our proposed method is able to generate realistic word image samples from different writer styles, by using class index styles and text content prompts without the need of adversarial training, writer recognition, or text recognition. We gauge system performance with the Fréchet Inception Distance, writer recognition accuracy, and writer retrieval. We show that the proposed model produces samples that are aesthetically pleasing, help boosting text recognition performance, and get similar writer retrieval score as real data.
APA:
Nikolaidou, K., Retsinas, G., Christlein, V., Seuret, M., Sfikas, G., Barney Smith, E.,... Liwicki, M. (2023). WordStylist: Styled Verbatim Handwritten Text Generation with Latent Diffusion Models. In Lecture Notes in Computer Science (pp. 384-401). San José, CA, US: Cham: Springer.
MLA:
Nikolaidou, Konstantina, et al. "WordStylist: Styled Verbatim Handwritten Text Generation with Latent Diffusion Models." Proceedings of the International Conference on Document Analysis and Recognition, San José, CA Cham: Springer, 2023. 384-401.
BibTeX: Download