We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Stochastic Neural Networks with Monotonic Activation Functions

Abstract: We propose a Laplace approximation that creates a stochastic unit from any smooth monotonic activation function, using only Gaussian noise. This paper investigates the application of this stochastic approximation in training a family of Restricted Boltzmann Machines (RBM) that are closely linked to Bregman divergences. This family, that we call exponential family RBM (Exp-RBM), is a subset of the exponential family Harmoniums that expresses family members through a choice of smooth monotonic non-linearity for each neuron. Using contrastive divergence along with our Gaussian approximation, we show that Exp-RBM can learn useful representations using novel stochastic units.
Comments: AISTATS 2016
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
Cite as: arXiv:1601.00034 [stat.ML]
  (or arXiv:1601.00034v4 [stat.ML] for this version)

Submission history

From: Siamak Ravanbakhsh [view email]
[v1] Fri, 1 Jan 2016 00:47:29 GMT (3701kb,D)
[v2] Thu, 14 Apr 2016 15:38:53 GMT (3701kb,D)
[v3] Thu, 21 Jul 2016 15:52:02 GMT (3701kb,D)
[v4] Fri, 22 Jul 2016 17:38:18 GMT (3701kb,D)

Link back to: arXiv, form interface, contact.