Assessing the Performance of Remaining Time Prediction Methods for Business Processes

Roider J, Nguyen A, Zanca D, Eskofier B (2024)

Publication Language: English

Publication Type: Journal article, Original article

Publication year: 2024

Journal

IEEE Access Institute of Electrical and Electronics Engineers (IEEE)

Pages Range: 1-19

URI: https://ieeexplore.ieee.org/abstract/document/10679137

DOI: 10.1109/ACCESS.2024.3459648

Open Access Link: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10679137

Abstract

The prediction of the remaining time for business processes is a major task in predictive process monitoring (PPM). In the last years, various machine learning methods were introduced which reduced error levels steadily. However, the commonly applied metric for optimization and evaluation, the Mean Absolute Error (MAE), has limitations regarding its interpretability. In this work we introduce and evaluate the normalized Mean Absolute Error (nMAE) as an interpretable metric for model evaluation. It accounts for different kinds of label shifts, which are a special type of concept drift that can distort remaining time results. We investigate these concepts in a thorough benchmark study and use them to assess the current state of remaining time prediction for business processes. This includes the evaluation of four different baseline models, identifying the most accurate one. Furthermore, our study compares three different state-of-the-art methods, namely XGBoost, DA-LSTM, and PGT-Net. In contrary to prior studies we find that there is no significant difference in the performance between these models. Additionally, using the nMAE as evaluation metric we find that these models do not perform reasonably well on a range of event logs. Initial ideas for this behaviour are discussed and consolidated along with other findings from the case study into a comprehensive list motivating future research directions.

Authors with CRIS profile

Johannes Roider Machine Learning and Data Analytics Lab An Nguyen Machine Learning and Data Analytics Lab Dario Zanca Machine Learning and Data Analytics Lab Björn Eskofier Machine Learning and Data Analytics Lab

How to cite

APA:

Roider, J., Nguyen, A., Zanca, D., & Eskofier, B. (2024). Assessing the Performance of Remaining Time Prediction Methods for Business Processes. IEEE Access, 1-19. https://doi.org/10.1109/ACCESS.2024.3459648

MLA:

Roider, Johannes, et al. "Assessing the Performance of Remaining Time Prediction Methods for Business Processes." IEEE Access (2024): 1-19.

BibTeX: Download