Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: An Information-Theoretic Framework for Fast and Robust Unsupervised Learning via Neural Population Infomax
(Submitted on 7 Nov 2016 (v1), last revised 10 Mar 2017 (this version, v4))
Abstract: A framework is presented for unsupervised learning of representations based on infomax principle for large-scale neural populations. We use an asymptotic approximation to the Shannon's mutual information for a large neural population to demonstrate that a good initial approximation to the global information-theoretic optimum can be obtained by a hierarchical infomax method. Starting from the initial solution, an efficient algorithm based on gradient descent of the final objective function is proposed to learn representations from the input datasets, and the method works for complete, overcomplete, and undercomplete bases. As confirmed by numerical experiments, our method is robust and highly efficient for extracting salient features from input datasets. Compared with the main existing methods, our algorithm has a distinct advantage in both the training speed and the robustness of unsupervised representation learning. Furthermore, the proposed method is easily extended to the supervised or unsupervised model for training deep structure networks.
Submission history
From: Wentao Huang [view email][v1] Mon, 7 Nov 2016 04:17:28 GMT (4776kb,D)
[v2] Thu, 19 Jan 2017 17:53:31 GMT (4782kb,D)
[v3] Mon, 6 Feb 2017 17:11:34 GMT (4781kb,D)
[v4] Fri, 10 Mar 2017 16:41:16 GMT (4782kb,D)
Link back to: arXiv, form interface, contact.