Current browse context:
cs.NE
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Stochastic Neural Networks with Monotonic Activation Functions
(Submitted on 1 Jan 2016 (v1), last revised 22 Jul 2016 (this version, v4))
Abstract: We propose a Laplace approximation that creates a stochastic unit from any smooth monotonic activation function, using only Gaussian noise. This paper investigates the application of this stochastic approximation in training a family of Restricted Boltzmann Machines (RBM) that are closely linked to Bregman divergences. This family, that we call exponential family RBM (Exp-RBM), is a subset of the exponential family Harmoniums that expresses family members through a choice of smooth monotonic non-linearity for each neuron. Using contrastive divergence along with our Gaussian approximation, we show that Exp-RBM can learn useful representations using novel stochastic units.
Submission history
From: Siamak Ravanbakhsh [view email][v1] Fri, 1 Jan 2016 00:47:29 GMT (3701kb,D)
[v2] Thu, 14 Apr 2016 15:38:53 GMT (3701kb,D)
[v3] Thu, 21 Jul 2016 15:52:02 GMT (3701kb,D)
[v4] Fri, 22 Jul 2016 17:38:18 GMT (3701kb,D)
Link back to: arXiv, form interface, contact.