Provable Boolean interaction recovery from tree ensemble obtained via random forests

Behr M, Wang Y, Li X, Yu B (2022)

Publication Type: Journal article

Publication year: 2022

Journal

Proceedings of the National Academy of Sciences of the United States of America National Academy of Sciences

Book Volume: 119

Article Number: e2118636119

Journal Issue: 22

DOI: 10.1073/pnas.2118636119

Abstract

Random Forests (RFs) are at the cutting edge of supervised machine learning in terms of prediction performance, especially in genomics. Iterative RFs (iRFs) use a tree ensemble from iteratively modified RFs to obtain predictive and stable nonlinear or Boolean interactions of features. They have shown great promise for Boolean biological interaction discovery that is central to advancing functional genomics and precision medicine. However, theoretical studies into how tree-based methods discover Boolean feature interactions are missing. Inspired by the thresholding behavior in many biological processes, we first introduce a discontinuous nonlinear regression model, called the "Locally Spiky Sparse" (LSS) model. Specifically, the LSS model assumes that the regression function is a linear combination of piecewise constant Boolean interaction terms. Given an RF tree ensemble, we define a quantity called "Depth-Weighted Prevalence" (DWP) for a set of signed features S±. Intuitively speaking, DWP(S±) measures how frequently features in S± appear together in an RF tree ensemble.We prove that,with high probability,DWP(S±) attains a universal upper bound that does not involve any model coefficients, if and only if S± corresponds to a union of Boolean interactions under the LSS model. Consequentially, we show that a theoretically tractable version of the iRF procedure, called LSSFind, yields consistent interaction discovery under the LSS model as the sample size goes to infinity. Finally, simulation results show that LSSFind recovers the interactions under the LSS model, even when some assumptions are violated.

Involved external institutions

University of California, Berkeley

United States (USA) (US)

How to cite

APA:

Behr, M., Wang, Y., Li, X., & Yu, B. (2022). Provable Boolean interaction recovery from tree ensemble obtained via random forests. Proceedings of the National Academy of Sciences of the United States of America, 119(22). https://dx.doi.org/10.1073/pnas.2118636119

MLA:

Behr, Merle, et al. "Provable Boolean interaction recovery from tree ensemble obtained via random forests." Proceedings of the National Academy of Sciences of the United States of America 119.22 (2022).

BibTeX: Download