Current browse context:
stat
Change to browse by:
References & Citations
Mathematics > Probability
Title: Pattern reconstruction with restricted Boltzmann machines
(Submitted on 14 May 2022 (v1), last revised 27 Oct 2023 (this version, v3))
Abstract: Restricted Boltzmann machines are energy models made of a visible and a hidden layer. We identify an effective energy function describing the zero-temperature landscape on the visible units and depending only on the tail behaviour of the hidden layer prior distribution. Studying the location of the local minima of such an energy function, we show that the ability of a restricted Boltzmann machine to reconstruct a random pattern depends indeed only on the tail of the hidden prior distribution. We find that hidden priors with strictly super-Gaussian tails give only a logarithmic loss in pattern retrieval, while an efficient retrieval is much harder with hidden units with strictly sub-Gaussian tails; if the hidden prior has Gaussian tails, the retrieval capability is determined by the number of hidden units (as in the Hopfield model).
Submission history
From: Giuseppe Genovese [view email][v1] Sat, 14 May 2022 16:00:21 GMT (21kb)
[v2] Wed, 5 Oct 2022 19:07:50 GMT (28kb)
[v3] Fri, 27 Oct 2023 18:13:49 GMT (25kb)
Link back to: arXiv, form interface, contact.