Current browse context:
stat
Change to browse by:
References & Citations
Condensed Matter > Disordered Systems and Neural Networks
Title: Dense Hebbian neural networks: a replica symmetric picture of unsupervised learning
(Submitted on 25 Nov 2022 (v1), last revised 2 Jul 2023 (this version, v2))
Abstract: We consider dense, associative neural-networks trained with no supervision and we investigate their computational capabilities analytically, via a statistical-mechanics approach, and numerically, via Monte Carlo simulations. In particular, we obtain a phase diagram summarizing their performance as a function of the control parameters such as the quality and quantity of the training dataset and the network storage, valid in the limit of large network size and structureless datasets. Moreover, we establish a bridge between macroscopic observables standardly used in statistical mechanics and loss functions typically used in the machine learning. As technical remarks, from the analytic side, we implement large deviations and stability analysis within Guerra's interpolation to tackle the not-Gaussian distributions involved in the post-synaptic potentials while, from the computational counterpart, we insert Plefka approximation in the Monte Carlo scheme, to speed up the evaluation of the synaptic tensors, overall obtaining a novel and broad approach to investigate neural networks in general.
Submission history
From: Daniele Lotito [view email][v1] Fri, 25 Nov 2022 12:40:06 GMT (3198kb,D)
[v2] Sun, 2 Jul 2023 14:41:24 GMT (5365kb,D)
Link back to: arXiv, form interface, contact.