Identifying consistent statements about numerical data with dispersion-corrected subgroup discovery

Boley M, Goldsmith BR, Ghiringhelli LM, Vreeken J (2017)

Publication Type: Journal article

Publication year: 2017

Journal

Data Mining and Knowledge Discovery

Book Volume: 31

Pages Range: 1391-1418

Journal Issue: 5

DOI: 10.1007/s10618-017-0520-3

Abstract

Existing algorithms for subgroup discovery with numerical targets do not optimize the error or target variable dispersion of the groups they find. This often leads to unreliable or inconsistent statements about the data, rendering practical applications, especially in scientific domains, futile. Therefore, we here extend the optimistic estimator framework for optimal subgroup discovery to a new class of objective functions: we show how tight estimators can be computed efficiently for all functions that are determined by subgroup size (non-decreasing dependence), the subgroup median value, and a dispersion measure around the median (non-increasing dependence). In the important special case when dispersion is measured using the mean absolute deviation from the median, this novel approach yields a linear time algorithm. Empirical evaluation on a wide range of datasets shows that, when used within branch-and-bound search, this approach is highly efficient and indeed discovers subgroups with much smaller errors.

Authors with CRIS profile

Luca Ghiringhelli

Involved external institutions

Max Planck Institute for Informatics / Max-Planck-Institut für Informatik

Germany (DE) Fritz-Haber-Institut der Max-Planck-Gesellschaft (FHI)

Germany (DE)

How to cite

APA:

Boley, M., Goldsmith, B.R., Ghiringhelli, L.M., & Vreeken, J. (2017). Identifying consistent statements about numerical data with dispersion-corrected subgroup discovery. Data Mining and Knowledge Discovery, 31(5), 1391-1418. https://doi.org/10.1007/s10618-017-0520-3

MLA:

Boley, Mario, et al. "Identifying consistent statements about numerical data with dispersion-corrected subgroup discovery." Data Mining and Knowledge Discovery 31.5 (2017): 1391-1418.

BibTeX: Download