Maximising the Utility of Validation Sets for Imbalanced Noisy-label Meta-learning

Hoang DA, Nguyen C, Belagiannis V, Do TT, Carneiro G (2025)

Publication Type: Journal article

Publication year: 2025

Journal

Transactions on Machine Learning Research Microtome Publishing

Book Volume: 2025-March

Abstract

Meta-learning is an effective method to handle imbalanced and noisy-label learning, but it generally depends on a clean validation set. Unfortunately, this validation set has poor scalability when the number of classes increases, as traditionally these samples need to be randomly selected, manually labelled and balanced-distributed. This problem therefore has motivated the development of meta-learning methods to automatically select validation samples that are likely to have clean labels and balanced class distribution. Unfortunately, a common missing point of existing meta-learning methods for noisy label learning is the lack of consideration for data informativeness when constructing the validation set. The construction of an informative validation set requires hard samples, i.e., samples that the model has low confident prediction, but these samples are more likely to be noisy, which can degrade the meta reweighting process. Therefore, the balance between sample informative-ness and cleanness is an important criteria for validation set optimization. In this paper, we propose new criteria to characterise the utility of such meta-learning validation sets, based on: 1) sample informativeness; 2) balanced class distribution; and 3) label cleanliness. We also introduce a new imbalanced noisy-label meta-learning (INOLML) algorithm that automatically builds a validation set by maximising such utility criteria. The proposed method shows state-of-the-art (SOTA) results compared to previous meta-learning and noisy-label learning approaches on several noisy-label learning benchmarks.

Authors with CRIS profile

Vasileios Belagiannis Professur für Maschinelles Lernen in der Signalverarbeitung

Involved external institutions

Monash University AU Australia (AU) University of Surrey GB United Kingdom (GB)

How to cite

APA:

Hoang, D.A., Nguyen, C., Belagiannis, V., Do, T.T., & Carneiro, G. (2025). Maximising the Utility of Validation Sets for Imbalanced Noisy-label Meta-learning. Transactions on Machine Learning Research, 2025-March.

MLA:

Hoang, Dung Anh, et al. "Maximising the Utility of Validation Sets for Imbalanced Noisy-label Meta-learning." Transactions on Machine Learning Research 2025-March (2025).

BibTeX: Download