Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Structured Sparsification of Gated Recurrent Neural Networks
(Submitted on 13 Nov 2019)
Abstract: Recently, a lot of techniques were developed to sparsify the weights of neural networks and to remove networks' structure units, e.g. neurons. We adjust the existing sparsification approaches to the gated recurrent architectures. Specifically, in addition to the sparsification of weights and neurons, we propose sparsifying the preactivations of gates. This makes some gates constant and simplifies LSTM structure. We test our approach on the text classification and language modeling tasks. We observe that the resulting structure of gate sparsity depends on the task and connect the learned structure to the specifics of the particular tasks. Our method also improves neuron-wise compression of the model in most of the tasks.
Submission history
From: Ekaterina Lobacheva Ms [view email][v1] Wed, 13 Nov 2019 16:26:22 GMT (7714kb,D)
Link back to: arXiv, form interface, contact.