Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Soft Weight-Sharing for Neural Network Compression
(Submitted on 13 Feb 2017 (v1), last revised 9 May 2017 (this version, v2))
Abstract: The success of deep learning in numerous application domains created the de- sire to run and train them on mobile devices. This however, conflicts with their computationally, memory and energy intense nature, leading to a growing interest in compression. Recent work by Han et al. (2015a) propose a pipeline that involves retraining, pruning and quantization of neural network weights, obtaining state-of-the-art compression rates. In this paper, we show that competitive compression rates can be achieved by using a version of soft weight-sharing (Nowlan & Hinton, 1992). Our method achieves both quantization and pruning in one simple (re-)training procedure. This point of view also exposes the relation between compression and the minimum description length (MDL) principle.
Submission history
From: Karen Ullrich [view email][v1] Mon, 13 Feb 2017 22:54:18 GMT (4285kb,D)
[v2] Tue, 9 May 2017 14:05:43 GMT (4285kb,D)
Link back to: arXiv, form interface, contact.