LUMOS: Self-supervised Pretraining for Multimodal Task Load Estimation in Adaptive Instructional Systems
Oppelt MP, Lang-Richter NR, Eskofier B (2026)
Publication Type: Conference contribution
Publication year: 2026
Journal
Publisher: Springer Science and Business Media Deutschland GmbH
Book Volume: 16737 LNCS
Pages Range: 101-120
Conference Proceedings Title: Lecture Notes in Computer Science
Event location: Montreal, QC, CAN
ISBN: 9783032300140
DOI: 10.1007/978-3-032-30015-7_7
Abstract
Accurate task load (TL) detection is essential for adaptive instructional systems (AIS) that tailor instruction to learner states, yet progress is hindered by costly data collection and scarce, noisy annotations. Recordings often contain substantial unlabeled segments, like rest periods, training phases, questionnaire completion, that supervised methods cannot exploit. We present LUMOS (Learning from Unlabeled Multimodal Observations in Task Load Detection Systems), a self-supervised learning framework that leverages these unlabeled multimodal physiological and behavioral signals to learn transferable representations for TL estimation. LUMOS employs discriminative self-supervised objectives, including MoCo, SimCLR, SwAV, and Barlow Twins, with modality-aware augmentations across electrocardiography, photoplethysmography, electrodermal activity, respiration, electromyography, skin temperature, pupil diameter, and eye movement data. We evaluate LUMOS on 96 sessions spanning n-back tests, driving simulation, and gaming tasks (Overcooked! 2, Hogwarts Legacy) using ConvNeXt, xLSTM, and Transformer backbones. MoCo pretrained models achieve the highest accuracy (0.79 ± 0.01), outperforming fully supervised baselines (0.72 ± 0.03) and demonstrating robust generalization across domains. Gains are most pronounced in low label regimes, where self-supervised initialization consistently outperforms training from scratch. Our results demonstrate that contrastive pretraining on unlabeled segments reduces annotation requirements and improves generalization, providing a practical foundation for TL-aware AIS applications.
Authors with CRIS profile
Maximilian Oppelt
Björn Eskofier Department Artificial Intelligence in Biomedical Engineering (AIBE)
Björn Eskofier Department Artificial Intelligence in Biomedical Engineering (AIBE)
Involved external institutions
Germany (DE)How to cite
APA:
Oppelt, M.P., Lang-Richter, N.R., & Eskofier, B. (2026). LUMOS: Self-supervised Pretraining for Multimodal Task Load Estimation in Adaptive Instructional Systems. In Robert A. Sottilare, Jessica Schwarz (Eds.), Lecture Notes in Computer Science (pp. 101-120). Montreal, QC, CAN: Springer Science and Business Media Deutschland GmbH.
MLA:
Oppelt, Maximilian P., Nadine R. Lang-Richter, and Björn Eskofier. "LUMOS: Self-supervised Pretraining for Multimodal Task Load Estimation in Adaptive Instructional Systems." Proceedings of the 8th International Conference, Adaptive Instructional Systems, AIS 2026, held as Part of the 28th HCI International Conference, HCII 2026, Montreal, QC, CAN Ed. Robert A. Sottilare, Jessica Schwarz, Springer Science and Business Media Deutschland GmbH, 2026. 101-120.
BibTeX: Download