Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Stochastic Neural Networks with Monotonic Activation Functions
(Submitted on 1 Jan 2016 (this version), latest version 22 Jul 2016 (v4))
Abstract: We propose a Laplace approximation that closely links any smooth monotonic activation function to its stochastic counterpart using Gaussian noise. We investigate the application of this approximation in training a family of Restricted Boltzmann Machines (RBM) that are closely linked to Bregman divergences. This family -- exponential family RBM (Exp-RBM) -- is a subset of the exponential family Harmoniums that expresses family members through a choice of smooth monotonic non-linearities for neurons. Using contrastive divergence along with our Gaussian approximation, we show that Exp-RBMs can learn useful representations using novel stochastic units.
Submission history
From: Siamak Ravanbakhsh [view email][v1] Fri, 1 Jan 2016 00:47:29 GMT (3701kb,D)
[v2] Thu, 14 Apr 2016 15:38:53 GMT (3701kb,D)
[v3] Thu, 21 Jul 2016 15:52:02 GMT (3701kb,D)
[v4] Fri, 22 Jul 2016 17:38:18 GMT (3701kb,D)
Link back to: arXiv, form interface, contact.