We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.DC

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Distributed, Parallel, and Cluster Computing

Title: Railgun: streaming windows for mission critical systems

Abstract: Some mission critical systems, such as fraud detection, require accurate, real-time metrics over long time windows on applications that demand high throughputs and low latencies. As these applications need to run "forever", cope with large and spiky data loads, they further require to be run in a distributed setting. Unsurprisingly, we are unaware of any distributed streaming system that provides all those properties. Instead, existing systems take large simplifications, such as implementing sliding windows as a fixed set of partially overlapping windows, jeopardizing metric accuracy (violating financial regulator rules) or latency (breaching service agreements).
In this paper, we propose Railgun, a fault-tolerant, elastic, and distributed streaming system supporting real-time sliding windows for scenarios requiring high loads and millisecond-level latencies. We benchmarked an initial prototype of Railgun using real data, showing significant lower latency than Flink, and low memory usage, independent of window size.
Comments: Previously submitted to CIDR 2021
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Databases (cs.DB)
ACM classes: C.3; H.3.4; H.2.4
Cite as: arXiv:2009.00361 [cs.DC]
  (or arXiv:2009.00361v3 [cs.DC] for this version)

Submission history

From: Ana Sofia Gomes [view email]
[v1] Tue, 1 Sep 2020 11:33:19 GMT (307kb,D)
[v2] Fri, 4 Sep 2020 11:00:01 GMT (299kb,D)
[v3] Tue, 10 Nov 2020 10:08:03 GMT (299kb,D)

Link back to: arXiv, form interface, contact.