References & Citations
Computer Science > Data Structures and Algorithms
Title: On the Optimal Time/Space Tradeoff for Hash Tables
(Submitted on 31 Oct 2021 (v1), last revised 4 Nov 2021 (this version, v2))
Abstract: For nearly six decades, the central open question in the study of hash tables has been to determine the optimal achievable tradeoff curve between time and space. State-of-the-art hash tables offer the following guarantee: If keys/values are Theta(log n) bits each, then it is possible to achieve constant-time insertions/deletions/queries while wasting only O(loglog n) bits of space per key when compared to the information-theoretic optimum. Even prior to this bound being achieved, the target of O(loglog n) wasted bits per key was known to be a natural end goal, and was proven to be optimal for a number of closely related problems (e.g., stable hashing, dynamic retrieval, and dynamically-resized filters).
This paper shows that O(loglog n) wasted bits per key is not the end of the line for hashing. In fact, for any k \in [log* n], it is possible to achieve O(k)-time insertions/deletions, O(1)-time queries, and O(\log^{(k)} n) wasted bits per key (all with high probability in n). This means that, each time we increase insertion/deletion time by an \emph{additive constant}, we reduce the wasted bits per key \emph{exponentially}. We further show that this tradeoff curve is the best achievable by any of a large class of hash tables, including any hash table designed using the current framework for making constant-time hash tables succinct.
Submission history
From: Martín Farach-Colton [view email][v1] Sun, 31 Oct 2021 21:58:06 GMT (54kb)
[v2] Thu, 4 Nov 2021 00:16:29 GMT (55kb)
Link back to: arXiv, form interface, contact.