Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound
(Submitted on 23 Jun 2021 (v1), last revised 19 Oct 2021 (this version, v2))
Abstract: We investigate a stochastic counterpart of majority votes over finite ensembles of classifiers, and study its generalization properties. While our approach holds for arbitrary distributions, we instantiate it with Dirichlet distributions: this allows for a closed-form and differentiable expression for the expected risk, which then turns the generalization bound into a tractable training objective. The resulting stochastic majority vote learning algorithm achieves state-of-the-art accuracy and benefits from (non-vacuous) tight generalization bounds, in a series of numerical experiments when compared to competing algorithms which also minimize PAC-Bayes objectives -- both with uninformed (data-independent) and informed (data-dependent) priors.
Submission history
From: Valentina Zantedeschi Dr [view email][v1] Wed, 23 Jun 2021 16:57:23 GMT (1291kb,D)
[v2] Tue, 19 Oct 2021 17:07:20 GMT (1467kb,D)
Link back to: arXiv, form interface, contact.