Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Neural Bootstrapper
(Submitted on 2 Oct 2020 (v1), last revised 13 Dec 2021 (this version, v4))
Abstract: Bootstrapping has been a primary tool for ensemble and uncertainty quantification in machine learning and statistics. However, due to its nature of multiple training and resampling, bootstrapping deep neural networks is computationally burdensome; hence it has difficulties in practical application to the uncertainty estimation and related tasks. To overcome this computational bottleneck, we propose a novel approach called \emph{Neural Bootstrapper} (NeuBoots), which learns to generate bootstrapped neural networks through single model training. NeuBoots injects the bootstrap weights into the high-level feature layers of the backbone network and outputs the bootstrapped predictions of the target, without additional parameters and the repetitive computations from scratch. We apply NeuBoots to various machine learning tasks related to uncertainty quantification, including prediction calibrations in image classification and semantic segmentation, active learning, and detection of out-of-distribution samples. Our empirical results show that NeuBoots outperforms other bagging based methods under a much lower computational cost without losing the validity of bootstrapping.
Submission history
From: Sungbin Lim [view email][v1] Fri, 2 Oct 2020 15:30:04 GMT (2152kb,D)
[v2] Wed, 31 Mar 2021 02:37:47 GMT (3927kb,D)
[v3] Wed, 27 Oct 2021 07:11:08 GMT (6568kb,D)
[v4] Mon, 13 Dec 2021 17:35:06 GMT (6542kb,D)
Link back to: arXiv, form interface, contact.