We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.DS

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Computer Science > Data Structures and Algorithms

Title: Relative Error Streaming Quantiles

Abstract: Estimating ranks, quantiles, and distributions over streaming data is a central task in data analysis and monitoring. Given a stream of $n$ items from a data universe equipped with a total order, the task is to compute a sketch (data structure) of size polylogarithmic in $n$. Given the sketch and a query item $y$, one should be able to approximate its rank in the stream, i.e., the number of stream elements smaller than or equal to $y$.
Most works to date focused on additive $\varepsilon n$ error approximation, culminating in the KLL sketch that achieved optimal asymptotic behavior. This paper investigates multiplicative $(1\pm\varepsilon)$-error approximations to the rank. Practical motivation for multiplicative error stems from demands to understand the tails of distributions, and hence for sketches to be more accurate near extreme values.
The most space-efficient algorithms due to prior work store either $O(\log(\varepsilon^2 n)/\varepsilon^2)$ or $O(\log^3(\varepsilon n)/\varepsilon)$ universe items. We present a randomized sketch storing $O(\log^{1.5}(\varepsilon n)/\varepsilon)$ items that can $(1\pm\varepsilon)$-approximate the rank of each universe item with high constant probability; this space bound is within an $O(\sqrt{\log(\varepsilon n)})$ factor of optimal. Our algorithm does not require prior knowledge of the stream length and is fully mergeable, rendering it suitable for parallel and distributed computing environments.
Comments: Final version of the paper to appear in Journal of the ACM. Compared to the previous version, we removed any restrictions on the accuracy parameters in the main result and thoroughly revised the paper. 48 pages, 2 figures
Subjects: Data Structures and Algorithms (cs.DS)
ACM classes: F.2.2
DOI: 10.1145/3452021.3458323
Cite as: arXiv:2004.01668 [cs.DS]
  (or arXiv:2004.01668v4 [cs.DS] for this version)

Submission history

From: Pavel Veselý [view email]
[v1] Fri, 3 Apr 2020 16:44:34 GMT (286kb,D)
[v2] Fri, 17 Jul 2020 15:52:25 GMT (126kb,D)
[v3] Fri, 28 May 2021 16:38:57 GMT (132kb,D)
[v4] Thu, 24 Aug 2023 11:07:09 GMT (137kb,D)

Link back to: arXiv, form interface, contact.