Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Multiscale Residual Mixture of PCA: Dynamic Dictionaries for Optimal Basis Learning
(Submitted on 18 Jul 2017)
Abstract: In this paper we are interested in the problem of learning an over-complete basis and a methodology such that the reconstruction or inverse problem does not need optimization. We analyze the optimality of the presented approaches, their link to popular already known techniques s.a. Artificial Neural Networks,k-means or Oja's learning rule. Finally, we will see that one approach to reach the optimal dictionary is a factorial and hierarchical approach. The derived approach lead to a formulation of a Deep Oja Network. We present results on different tasks and present the resulting very efficient learning algorithm which brings a new vision on the training of deep nets. Finally, the theoretical work shows that deep frameworks are one way to efficiently have over-complete (combinatorially large) dictionary yet allowing easy reconstruction. We thus present the Deep Residual Oja Network (DRON). We demonstrate that a recursive deep approach working on the residuals allow exponential decrease of the error w.r.t. the depth.
Submission history
From: Randall Balestriero [view email][v1] Tue, 18 Jul 2017 19:56:05 GMT (1782kb,D)
Link back to: arXiv, form interface, contact.