We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ME

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Methodology

Title: An information criterion for automatic gradient tree boosting

Abstract: An information theoretic approach to learning the complexity of classification and regression trees and the number of trees in gradient tree boosting is proposed. The optimism (test loss minus training loss) of the greedy leaf splitting procedure is shown to be the maximum of a Cox-Ingersoll-Ross process, from which a generalization-error based information criterion is formed. The proposed procedure allows fast local model selection without cross validation based hyper parameter tuning, and hence efficient and automatic comparison among the large number of models performed during each boosting iteration. Relative to xgboost, speedups on numerical experiments ranges from around 10 to about 1400, at similar predictive-power measured in terms of test-loss.
Subjects: Methodology (stat.ME); Computation (stat.CO); Machine Learning (stat.ML)
Cite as: arXiv:2008.05926 [stat.ME]
  (or arXiv:2008.05926v1 [stat.ME] for this version)

Submission history

From: Berent Ånund Strømnes Lunde [view email]
[v1] Thu, 13 Aug 2020 14:24:27 GMT (394kb,D)

Link back to: arXiv, form interface, contact.