Current browse context:
cs
Change to browse by:
References & Citations
Physics > Data Analysis, Statistics and Probability
Title: Emergence of Compositional Representations in Restricted Boltzmann Machines
(Submitted on 21 Nov 2016 (v1), last revised 2 Mar 2017 (this version, v2))
Abstract: Extracting automatically the complex set of features composing real high-dimensional data is crucial for achieving high performance in machine--learning tasks. Restricted Boltzmann Machines (RBM) are empirically known to be efficient for this purpose, and to be able to generate distributed and graded representations of the data. We characterize the structural conditions (sparsity of the weights, low effective temperature, nonlinearities in the activation functions of hidden units, and adaptation of fields maintaining the activity in the visible layer) allowing RBM to operate in such a compositional phase. Evidence is provided by the replica analysis of an adequate statistical ensemble of random RBMs and by RBM trained on the handwritten digits dataset MNIST.
Submission history
From: Jérôme Tubiana [view email][v1] Mon, 21 Nov 2016 12:46:25 GMT (651kb,D)
[v2] Thu, 2 Mar 2017 21:50:02 GMT (1214kb,D)
Link back to: arXiv, form interface, contact.