We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Streaming Bayesian inference: theoretical limits and mini-batch approximate message-passing

Abstract: In statistical learning for real-world large-scale data problems, one must often resort to "streaming" algorithms which operate sequentially on small batches of data. In this work, we present an analysis of the information-theoretic limits of mini-batch inference in the context of generalized linear models and low-rank matrix factorization. In a controlled Bayes-optimal setting, we characterize the optimal performance and phase transitions as a function of mini-batch size. We base part of our results on a detailed analysis of a mini-batch version of the approximate message-passing algorithm (Mini-AMP), which we introduce. Additionally, we show that this theoretical optimality carries over into real-data problems by illustrating that Mini-AMP is competitive with standard streaming algorithms for clustering.
Comments: 19 pages, 4 figures
Subjects: Machine Learning (stat.ML); Statistical Mechanics (cond-mat.stat-mech); Information Theory (cs.IT)
Journal reference: 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton), Monticello, IL, USA, 2017, pp. 1048-1055
DOI: 10.1109/ALLERTON.2017.8262853
Cite as: arXiv:1706.00705 [stat.ML]
  (or arXiv:1706.00705v1 [stat.ML] for this version)

Submission history

From: Andre Manoel [view email]
[v1] Fri, 2 Jun 2017 14:45:15 GMT (1610kb,D)

Link back to: arXiv, form interface, contact.