We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.AP

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Statistics > Applications

Title: BayesSummaryStatLM: An R package for Bayesian Linear Models for Big Data and Data Science

Abstract: Recent developments in data science and big data research have produced an abundance of large data sets that are too big to be analyzed in their entirety, due to limits on either computer memory or storage capacity. Here, we introduce our R package 'BayesSummaryStatLM' for Bayesian linear regression models with Markov chain Monte Carlo implementation that overcomes these limitations. Our Bayesian models use only summary statistics of data as input; these summary statistics can be calculated from subsets of big data and combined over subsets. Thus, complete data sets do not need to be read into memory in full, which removes any physical memory limitations of a user. Our package incorporates the R package 'ff' and its functions for reading in big data sets in chunks while simultaneously calculating summary statistics. We describe our Bayesian linear regression models, including several choices of prior distributions for unknown model parameters, and illustrate capabilities and features of our R package using both simulated and real data sets.
Comments: Updated URL in reference [12]; added to description of zero.intercept on p. 11; added minor clarifications throughout; results unchanged
Subjects: Applications (stat.AP); Computation (stat.CO)
Cite as: arXiv:1503.00635 [stat.AP]
  (or arXiv:1503.00635v2 [stat.AP] for this version)

Submission history

From: Erin Conlon [view email]
[v1] Mon, 2 Mar 2015 17:44:38 GMT (446kb)
[v2] Thu, 23 Apr 2015 23:01:04 GMT (443kb)

Link back to: arXiv, form interface, contact.