Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: FreezeOut: Accelerate Training by Progressively Freezing Layers
(Submitted on 15 Jun 2017 (v1), last revised 18 Jun 2017 (this version, v2))
Abstract: The early layers of a deep neural net have the fewest parameters, but take up the most computation. In this extended abstract, we propose to only train the hidden layers for a set portion of the training run, freezing them out one-by-one and excluding them from the backward pass. Through experiments on CIFAR, we empirically demonstrate that FreezeOut yields savings of up to 20% wall-clock time during training with 3% loss in accuracy for DenseNets, a 20% speedup without loss of accuracy for ResNets, and no improvement for VGG networks. Our code is publicly available at this https URL
Submission history
From: Andrew Brock [view email][v1] Thu, 15 Jun 2017 17:35:15 GMT (3224kb,D)
[v2] Sun, 18 Jun 2017 16:15:21 GMT (3273kb,D)
Link back to: arXiv, form interface, contact.