Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Random active path model of deep neural networks with diluted binary synapses
(Submitted on 2 May 2017 (v1), last revised 26 Sep 2018 (this version, v3))
Abstract: Deep learning has become a powerful and popular tool for a variety of machine learning tasks. However, it is challenging to understand the mechanism of deep learning from a theoretical perspective. In this work, we propose a random active path model to study collective properties of deep neural networks with binary synapses, under the removal perturbation of connections between layers. In the model, the path from input to output is randomly activated, and the corresponding input unit constrains the weights along the path into the form of a $p$-weight interaction glass model. A critical value of the perturbation is observed to separate a spin glass regime from a paramagnetic regime, with the transition being of the first order. The paramagnetic phase is conjectured to have a poor generalization performance.
Submission history
From: Haiping Huang [view email][v1] Tue, 2 May 2017 08:16:12 GMT (1345kb)
[v2] Fri, 18 May 2018 00:49:27 GMT (110kb,D)
[v3] Wed, 26 Sep 2018 13:09:33 GMT (153kb,D)
Link back to: arXiv, form interface, contact.