Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Stochastic Neural Networks with Monotonic Activation Functions
(Submitted on 1 Jan 2016 (v1), revised 21 Jul 2016 (this version, v3), latest version 22 Jul 2016 (v4))
Abstract: We propose a Laplace approximation that creates a stochastic unit from any smooth monotonic activation function, using only Gaussian noise. This paper investigates the application of this stochastic approximation in training a family of Restricted Boltzmann Machines (RBM) that are closely linked to Bregman divergences. This family, that we call exponential family RBM (Exp-RBM), is a subset of the exponential family Harmoniums that expresses family members through a choice of smooth monotonic non-linearity for each neuron. Using contrastive divergence along with our Gaussian approximation, we show that Exp-RBM can learn useful representations using novel stochastic units.
Submission history
From: Siamak Ravanbakhsh [view email][v1] Fri, 1 Jan 2016 00:47:29 GMT (3701kb,D)
[v2] Thu, 14 Apr 2016 15:38:53 GMT (3701kb,D)
[v3] Thu, 21 Jul 2016 15:52:02 GMT (3701kb,D)
[v4] Fri, 22 Jul 2016 17:38:18 GMT (3701kb,D)
Link back to: arXiv, form interface, contact.