Towards a statistically well-grounded evaluation of listening tests - Avoiding pitfalls, misuse, and misconceptions

Nagel F, Sporer T, Sedlmeier P (2010)

Publication Type: Conference contribution

Publication year: 2010

Book Volume: 1

Pages Range: 299-308

Conference Proceedings Title: 128th Audio Engineering Society Convention 2010

Event location: GBR

ISBN: 9781617387739

Abstract

Many recent publications in audio research present subjective evaluations of audio quality based on the Recommendation ITU-R BS.1534-1 (MUSHRA, MUltiple Stimuli with Hidden Reference and Anchor). This is a very welcome trend because it enables researchers to assess the implications of their developments. The evaluation of listening tests, however, sometimes suffers from an incomplete understanding of the underlying statistics. The present paper aims at identifying the causes for the pitfalls and misconceptions in MUSHRA evaluations. It exemplifies the impact of falsely used or even misused statistics. Subsequently, schemes for evaluating the listeners' judgments that are well-grounded on statistical considerations comprising an understanding of the concepts of statistical power and effect size are proposed.

Involved external institutions

Fraunhofer-Institut für Integrierte Schaltungen (IIS) DE Germany (DE) Fraunhofer-Institut für Digitale Medientechnologie IDMT DE Germany (DE) Technische Universität Chemnitz DE Germany (DE)

How to cite

APA:

Nagel, F., Sporer, T., & Sedlmeier, P. (2010). Towards a statistically well-grounded evaluation of listening tests - Avoiding pitfalls, misuse, and misconceptions. In 128th Audio Engineering Society Convention 2010 (pp. 299-308). GBR.

MLA:

Nagel, Frederik, Thomas Sporer, and Peter Sedlmeier. "Towards a statistically well-grounded evaluation of listening tests - Avoiding pitfalls, misuse, and misconceptions." Proceedings of the 128th Audio Engineering Society Convention 2010, GBR 2010. 299-308.

BibTeX: Download