Current browse context:
cond-mat.dis-nn
Change to browse by:
References & Citations
Condensed Matter > Disordered Systems and Neural Networks
Title: Mean-field inference methods for neural networks
(Submitted on 3 Nov 2019 (v1), last revised 5 Mar 2020 (this version, v2))
Abstract: Machine learning algorithms relying on deep neural networks recently allowed a great leap forward in artificial intelligence. Despite the popularity of their applications, the efficiency of these algorithms remains largely unexplained from a theoretical point of view. The mathematical description of learning problems involves very large collections of interacting random variables, difficult to handle analytically as well as numerically. This complexity is precisely the object of study of statistical physics. Its mission, originally pointed towards natural systems, is to understand how macroscopic behaviors arise from microscopic laws. Mean-field methods are one type of approximation strategy developed in this view. We review a selection of classical mean-field methods and recent progress relevant for inference in neural networks. In particular, we remind the principles of derivations of high-temperature expansions, the replica method and message passing algorithms, highlighting their equivalences and complementarities. We also provide references for past and current directions of research on neural networks relying on mean-field methods.
Submission history
From: Marylou Gabrié [view email][v1] Sun, 3 Nov 2019 14:04:57 GMT (2223kb,D)
[v2] Thu, 5 Mar 2020 12:13:23 GMT (1928kb,D)
Link back to: arXiv, form interface, contact.