We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.AP

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Statistics > Applications

Title: Big Data analytics. Three use cases with R, Python and Spark

Abstract: Management and analysis of big data are systematically associated with a data distributed architecture in the Hadoop and now Spark frameworks. This article offers an introduction for statisticians to these technologies by comparing the performance obtained by the direct use of three reference environments: R, Python Scikit-learn, Spark MLlib on three public use cases: character recognition, recommending films, categorizing products. As main result, it appears that, if Spark is very efficient for data munging and recommendation by collaborative filtering (non-negative factorization), current implementations of conventional learning methods (logistic regression, random forests) in MLlib or SparkML do not ou poorly compete habitual use of these methods (R, Python Scikit-learn) in an integrated or undistributed architecture
Comments: in French, Apprentissage Statistique et Donn{\'e}es Massives, Technip, 2017, Journ{\'e}es d'Etudes en Statistisque
Subjects: Applications (stat.AP); Machine Learning (cs.LG)
Cite as: arXiv:1609.09619 [stat.AP]
  (or arXiv:1609.09619v1 [stat.AP] for this version)

Submission history

From: Philippe Besse [view email]
[v1] Fri, 30 Sep 2016 07:35:49 GMT (278kb,D)

Link back to: arXiv, form interface, contact.