Current browse context:
cs.DC
Change to browse by:
References & Citations
Computer Science > Distributed, Parallel, and Cluster Computing
Title: A parallel workload has extreme variability
(Submitted on 13 Nov 2016 (v1), last revised 18 Nov 2016 (this version, v2))
Abstract: In both high-performance computing (HPC) environments and the public cloud, the duration of time to retrieve or save your results is simultaneously unpredictable and important to your over all resource budget. It is generally accepted ("Google: Taming the Long Latency Tail - When More Machines Equals Worse Results", Todd Hoff, highscalability.com 2012), but without a robust explanation, that identical parallel tasks do take different durations to complete -- a phenomena known as variability. This paper advances understanding of this topic. We carefully choose a model from which system-level complexity emerges that can be studied directly. We find that a generalized extreme value (GEV) model for variability naturally emerges. Using the public cloud, we find real-world observations have excellent agreement with our model. Since the GEV distribution is a limit distribution this suggests a universal property of parallel systems gated by the slowest communication element of some sort. Hence, this model is applicable to a variety of processing and IO tasks in parallel environments. These findings have important implications, ranging from characterizing ideal performance for parallel codes to detecting degraded behaviour at extreme scales.
Submission history
From: Richard Henwood Mr [view email][v1] Sun, 13 Nov 2016 18:18:36 GMT (51kb,D)
[v2] Fri, 18 Nov 2016 15:37:14 GMT (51kb,D)
Link back to: arXiv, form interface, contact.