This Paper Had the Smartest Reviewers - Flattery Detection Utilising an Audio-Textual Transformer-Based Approach

Christ L, Amiriparian S, Hawighorst F, Schill AK, Boutalikakis A, Graf-Vlachy L, König A, Schuller BW (2024)

Publication Type: Conference contribution

Publication year: 2024

Publisher: International Speech Communication Association

Pages Range: 3530-3534

Conference Proceedings Title: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Event location: Kos Island, GRC

DOI: 10.21437/Interspeech.2024-87

Abstract

Flattery is an important aspect of human communication that facilitates social bonding, shapes perceptions, and influences behaviour through strategic compliments and praise, leveraging the power of speech to build rapport effectively. Its automatic detection can thus enhance the naturalness of human-AI interactions. To meet this need, we present a novel audio textual dataset comprising 20 hours of speech and train machine learning models for automatic flattery detection. In particular, we employ pretrained AST, Wav2Vec2, and Whisper models for the speech modality, and Whisper TTS models combined with a RoBERTa text classifier for the textual modality. Subsequently, we build a multimodal classifier by combining text and audio representations. Evaluation on unseen test data demonstrates promising results, with Unweighted Average Recall scores reaching 82.46% in audio-only experiments, 85.97 % in text-only experiments, and 87.16 % using a multimodal approach.

Involved external institutions

Universität Augsburg DE Germany (DE) Technische Universität München (TUM) DE Germany (DE) Universität Passau DE Germany (DE) Technische Universität Dortmund DE Germany (DE)

How to cite

APA:

Christ, L., Amiriparian, S., Hawighorst, F., Schill, A.K., Boutalikakis, A., Graf-Vlachy, L.,... Schuller, B.W. (2024). This Paper Had the Smartest Reviewers - Flattery Detection Utilising an Audio-Textual Transformer-Based Approach. In Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH (pp. 3530-3534). Kos Island, GRC: International Speech Communication Association.

MLA:

Christ, Lukas, et al. "This Paper Had the Smartest Reviewers - Flattery Detection Utilising an Audio-Textual Transformer-Based Approach." Proceedings of the 25th Interspeech Conferece 2024, Kos Island, GRC International Speech Communication Association, 2024. 3530-3534.

BibTeX: Download