PASAL: Progress- and sparsity-aware loss balancing for heterogeneous dataset fusion

Gorges T, Scholz T, Saloman S, Zinnen M, Hoffmann J, Gourmelon N, Maier A, Hettenkofer S, Christlein V (2025)

Publication Type: Journal article

Publication year: 2025

Journal

Book Volume: 120

Pages Range: 103038

Article Number: 103038

DOI: 10.1016/j.inffus.2025.103038

Abstract

Machine learning has seen widespread application in many areas. Despite theoretical advancements, the demand for qualitative and extensive data foundations is increasing. Real-world datasets are often small and combining them is challenging due to the resulting sparsity and heterogeneity. Existing combination techniques merge datasets into a common space before training, causing drawbacks such as data loss and distortion of annotations. To address this, we fuse heterogeneous datasets by jointly training dataset-specific weighted sub-networks. Balancing losses from heterogeneous data sources is challenging, as current techniques are inadequate. We propose a novel progress- and sparsity-aware loss balancing method (PASAL), which adaptively balances sub-network losses based on individual learning progress and sparsity. As an example, we present the application of PASAL to the olfaction domain, where predicting smell properties based on molecular structure is difficult due to subjective impressions, typically limited data, and a lack of unified datasets. By evaluating PASAL on the DREAM Olfaction Prediction Challenge, we improve the current state-of-the-art method from a Z-Score of 9.92 to 10.10. Furthermore, by treating our AI as an annotator, we surpass human performance in the odor and pleasantness categories with statistical significance. Our findings are supported by a feature analysis, indicating that our heterogeneous combination methodology enhances odor prediction.

Authors with CRIS profile

Involved external institutions

Otto-von-Guericke-Universität Magdeburg Germany (DE) Fraunhofer-Institut für Integrierte Schaltungen (IIS) Germany (DE) Fraunhofer-Institut für Integrierte Systeme und Bauelementetechnologie (IISB) Germany (DE)

How to cite

APA:

Gorges, T., Scholz, T., Saloman, S., Zinnen, M., Hoffmann, J., Gourmelon, N.,... Christlein, V. (2025). PASAL: Progress- and sparsity-aware loss balancing for heterogeneous dataset fusion. Information Fusion, 120, 103038. https://doi.org/10.1016/j.inffus.2025.103038

MLA:

Gorges, Thomas, et al. "PASAL: Progress- and sparsity-aware loss balancing for heterogeneous dataset fusion." Information Fusion 120 (2025): 103038.

BibTeX: Download