Current browse context:
cs
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Gaussian Process Boosting
(Submitted on 6 Apr 2020 (v1), last revised 20 Sep 2022 (this version, v6))
Abstract: We introduce a novel way to combine boosting with Gaussian process and mixed effects models. This allows for relaxing, first, the zero or linearity assumption for the prior mean function in Gaussian process and grouped random effects models in a flexible non-parametric way and, second, the independence assumption made in most boosting algorithms. The former is advantageous for prediction accuracy and for avoiding model misspecifications. The latter is important for efficient learning of the fixed effects predictor function and for obtaining probabilistic predictions. Our proposed algorithm is also a novel solution for handling high-cardinality categorical variables in tree-boosting. In addition, we present an extension that scales to large data using a Vecchia approximation for the Gaussian process model relying on novel results for covariance parameter inference. We obtain increased prediction accuracy compared to existing approaches on multiple simulated and real-world data sets.
Submission history
From: Fabio Sigrist [view email][v1] Mon, 6 Apr 2020 13:19:54 GMT (44kb)
[v2] Wed, 23 Dec 2020 20:00:36 GMT (2159kb,D)
[v3] Fri, 16 Jul 2021 12:06:59 GMT (2160kb,D)
[v4] Tue, 23 Aug 2022 05:42:00 GMT (1528kb,D)
[v5] Fri, 26 Aug 2022 15:38:09 GMT (1528kb,D)
[v6] Tue, 20 Sep 2022 07:51:17 GMT (1532kb,D)
Link back to: arXiv, form interface, contact.