References & Citations
Computer Science > Distributed, Parallel, and Cluster Computing
Title: Performance Modeling and Vertical Autoscaling of Stream Joins
(Submitted on 11 May 2020 (this version), latest version 29 Nov 2021 (v2))
Abstract: Streaming analysis is widely used in cloud as well as edge infrastructures. In these contexts, fine-grained application performance can be based on accurate modeling of streaming operators. This is especially beneficial for computationally expensive operators like adaptive stream joins that, being very sensitive to rate-varying data streams, would otherwise require costly frequent monitoring.
We propose a dynamic model for the processing throughput and latency of adaptive stream joins that run with different parallelism degrees. The model is presented with progressive complexity, from a centralized non-deterministic up to a deterministic parallel stream join, describing how throughput and latency dynamics are influenced by various configuration parameters. The model is catalytic for understanding the behavior of stream joins against different system deployments, as we show with our model-based autoscaling methodology to change the parallelism degree of stream joins during the execution. Our thorough evaluation, for a broad spectrum of parameter, confirms the model can reliably predict throughput and latency metrics with a fairly high accuracy, with the median error in estimation ranging from approximately 0.1% to 6.5%, even for an overloaded system. Furthermore, we show that our model allows to efficiently control adaptive stream joins by estimating the needed resources solely based on the observed input load. In particular, we show it can be employed to enable efficient autoscaling, even when big changes in the input load happen frequently (in the realm of seconds).
Submission history
From: Vincenzo Gulisano [view email][v1] Mon, 11 May 2020 09:02:53 GMT (1908kb,D)
[v2] Mon, 29 Nov 2021 08:56:32 GMT (0kb,I)
Link back to: arXiv, form interface, contact.