We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Data-driven advice for interpreting local and global model predictions in bioinformatics problems

Abstract: Tree-based algorithms such as random forests and gradient boosted trees continue to be among the most popular and powerful machine learning models used across multiple disciplines. The conventional wisdom of estimating the impact of a feature in tree based models is to measure the \textit{node-wise reduction of a loss function}, which (i) yields only global importance measures and (ii) is known to suffer from severe biases. Conditional feature contributions (CFCs) provide \textit{local}, case-by-case explanations of a prediction by following the decision path and attributing changes in the expected output of the model to each feature along the path. However, Lundberg et al. pointed out a potential bias of CFCs which depends on the distance from the root of a tree. The by now immensely popular alternative, SHapley Additive exPlanation (SHAP) values appear to mitigate this bias but are computationally much more expensive. Here we contribute a thorough comparison of the explanations computed by both methods on a set of 164 publicly available classification problems in order to provide data-driven algorithm recommendations to current researchers. For random forests, we find extremely high similarities and correlations of both local and global SHAP values and CFC scores, leading to very similar rankings and interpretations. Analogous conclusions hold for the fidelity of using global feature importance scores as a proxy for the predictive power associated with each feature.
Comments: arXiv admin note: substantial text overlap with arXiv:2003.12043. text overlap with arXiv:1905.04610 by other authors
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Computation (stat.CO)
Cite as: arXiv:2108.06201 [stat.ML]
  (or arXiv:2108.06201v2 [stat.ML] for this version)

Submission history

From: Markus Loecher [view email]
[v1] Fri, 13 Aug 2021 12:41:39 GMT (101kb,D)
[v2] Thu, 30 Dec 2021 16:59:53 GMT (280kb,D)

Link back to: arXiv, form interface, contact.