Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Controlling Exploration Improves Training for Deep Neural Networks
(Submitted on 31 May 2016 (this version), latest version 28 Sep 2017 (v2))
Abstract: Stochastic optimization methods are widely used for training of deep neural networks. However, it is still a challenging research problem to achieve effective training by using stochastic optimization methods. This is due to the difficulties in finding good parameters on a loss function that have many saddle points. In this paper, we propose a stochastic optimization method called STDProp for effective training of deep neural networks. Its key idea is to effectively explore parameters on a complex surface of a loss function. We additionally develop momentum version of STDProp. While our approaches are easy to implement with high memory efficiency, it is more effective than other practical stochastic optimization methods for deep neural networks.
Submission history
From: Yasutoshi Ida [view email][v1] Tue, 31 May 2016 12:11:51 GMT (29kb)
[v2] Thu, 28 Sep 2017 10:24:50 GMT (48kb,D)
Link back to: arXiv, form interface, contact.