Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: SGLB: Stochastic Gradient Langevin Boosting
(Submitted on 20 Jan 2020 (v1), last revised 16 Jan 2022 (this version, v5))
Abstract: This paper introduces Stochastic Gradient Langevin Boosting (SGLB) - a powerful and efficient machine learning framework that may deal with a wide range of loss functions and has provable generalization guarantees. The method is based on a special form of the Langevin diffusion equation specifically designed for gradient boosting. This allows us to theoretically guarantee the global convergence even for multimodal loss functions, while standard gradient boosting algorithms can guarantee only local optimum. We also empirically show that SGLB outperforms classic gradient boosting when applied to classification tasks with 0-1 loss function, which is known to be multimodal.
Submission history
From: Liudmila Ostroumova Prokhorenkova [view email][v1] Mon, 20 Jan 2020 20:44:52 GMT (56kb)
[v2] Wed, 4 Mar 2020 14:45:51 GMT (46kb)
[v3] Mon, 6 Jul 2020 15:56:26 GMT (37kb)
[v4] Sun, 4 Jul 2021 19:03:26 GMT (47kb,D)
[v5] Sun, 16 Jan 2022 12:02:24 GMT (47kb,D)
Link back to: arXiv, form interface, contact.