We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

q-bio.NC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Quantitative Biology > Neurons and Cognition

Title: ODE-Inspired Analysis for the Biological Version of Oja's Rule in Solving Streaming PCA

Abstract: Oja's rule [Oja, Journal of mathematical biology 1982] is a well-known biologically-plausible algorithm using a Hebbian-type synaptic update rule to solve streaming principal component analysis (PCA). Computational neuroscientists have known that this biological version of Oja's rule converges to the top eigenvector of the covariance matrix of the input in the limit. However, prior to this work, it was open to prove any convergence rate guarantee.
In this work, we give the first convergence rate analysis for the biological version of Oja's rule in solving streaming PCA. Moreover, our convergence rate matches the information theoretical lower bound up to logarithmic factors and outperforms the state-of-the-art upper bound for streaming PCA. Furthermore, we develop a novel framework inspired by ordinary differential equations (ODE) to analyze general stochastic dynamics. The framework abandons the traditional step-by-step analysis and instead analyzes a stochastic dynamic in one-shot by giving a closed-form solution to the entire dynamic. The one-shot framework allows us to apply stopping time and martingale techniques to have a flexible and precise control on the dynamic. We believe that this general framework is powerful and should lead to effective yet simple analysis for a large class of problems with stochastic dynamics.
Comments: Accepted for presentation at the Conference on Learning Theory (COLT) 2020
Subjects: Neurons and Cognition (q-bio.NC); Data Structures and Algorithms (cs.DS); Machine Learning (cs.LG)
Cite as: arXiv:1911.02363 [q-bio.NC]
  (or arXiv:1911.02363v2 [q-bio.NC] for this version)

Submission history

From: Chi-Ning Chou [view email]
[v1] Mon, 4 Nov 2019 16:01:32 GMT (1025kb,D)
[v2] Wed, 17 Jun 2020 21:32:51 GMT (1027kb,D)

Link back to: arXiv, form interface, contact.