We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.DC

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Distributed, Parallel, and Cluster Computing

Title: Fault Tolerance for Stream Processing Engines

Abstract: Distributed Stream Processing Engines (DSPEs) target applications related to continuous computation, online machine learning and real-time query processing. DSPEs operate on high volume of data by applying lightweight operations on real-time and continuous streams. Such systems require clusters of hundreds of machine for their deployment. Streaming applications come with various requirements, i.e., low-latency, high throughput, scalability and high availability. In this survey, we study the fault tolerance problem for DSPEs. We discuss fault tolerance techniques that are used in modern stream processing engines that are Storm, S4, Samza, SparkStreaming and MillWheel. Further, we give insight on fault tolerance approaches that we categorize as active replication, passive replication and upstream backup. Finally, we discuss implications of the fault tolerance techniques for different streaming application requirements.
Comments: The survey is not complete and require major updates
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC)
Cite as: arXiv:1605.00928 [cs.DC]
  (or arXiv:1605.00928v3 [cs.DC] for this version)

Submission history

From: Muhammad Anis Uddin Nasir [view email]
[v1] Tue, 3 May 2016 14:30:05 GMT (653kb,D)
[v2] Wed, 19 Oct 2016 19:06:52 GMT (0kb,I)
[v3] Tue, 5 May 2020 09:40:16 GMT (0kb,I)

Link back to: arXiv, form interface, contact.