Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Single Shot Structured Pruning Before Training
(Submitted on 1 Jul 2020)
Abstract: We introduce a method to speed up training by 2x and inference by 3x in deep neural networks using structured pruning applied before training. Unlike previous works on pruning before training which prune individual weights, our work develops a methodology to remove entire channels and hidden units with the explicit aim of speeding up training and inference. We introduce a compute-aware scoring mechanism which enables pruning in units of sensitivity per FLOP removed, allowing even greater speed ups. Our method is fast, easy to implement, and needs just one forward/backward pass on a single batch of data to complete pruning before training begins.
Submission history
From: Joost van Amersfoort [view email][v1] Wed, 1 Jul 2020 11:27:37 GMT (1252kb,D)
Link back to: arXiv, form interface, contact.