Current browse context:
cs.LG
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Variational Depth Search in ResNets
(Submitted on 6 Feb 2020 (v1), last revised 1 Apr 2020 (this version, v4))
Abstract: One-shot neural architecture search allows joint learning of weights and network architecture, reducing computational cost. We limit our search space to the depth of residual networks and formulate an analytically tractable variational objective that allows for obtaining an unbiased approximate posterior over depths in one-shot. We propose a heuristic to prune our networks based on this distribution. We compare our proposed method against manual search over network depths on the MNIST, Fashion-MNIST, SVHN datasets. We find that pruned networks do not incur a loss in predictive performance, obtaining accuracies competitive with unpruned networks. Marginalising over depth allows us to obtain better-calibrated test-time uncertainty estimates than regular networks, in a single forward pass.
Submission history
From: Javier Antorán [view email][v1] Thu, 6 Feb 2020 16:00:03 GMT (3586kb,D)
[v2] Mon, 10 Feb 2020 16:12:58 GMT (3583kb,D)
[v3] Thu, 27 Feb 2020 10:58:12 GMT (3583kb,D)
[v4] Wed, 1 Apr 2020 17:59:13 GMT (3466kb,D)
Link back to: arXiv, form interface, contact.