We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.DC

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Distributed, Parallel, and Cluster Computing

Title: Time-Efficient and High-Quality Graph Partitioning for Graph Dynamic Scaling

Abstract: The dynamic scaling of distributed computations plays an important role in the utilization of elastic computational resources, such as the cloud. It enables the provisioning and de-provisioning of resources to match dynamic resource availability and demands. In the case of distributed graph processing, changing the number of the graph partitions while maintaining high partitioning quality imposes serious computational overheads as typically a time-consuming graph partitioning algorithm needs to execute each time repartitioning is required. In this paper, we propose a dynamic scaling method that can efficiently change the number of graph partitions while keeping its quality high. Our idea is based on two techniques: preprocessing and very fast edge partitioning, called graph edge ordering and chunk-based edge partitioning, respectively. The former converts the graph data into an ordered edge list in such a way that edges with high locality are closer to each other. The latter immediately divides the ordered edge list into an arbitrary number of high-quality partitions. The evaluation with the real-world billion-scale graphs demonstrates that our proposed approach significantly reduces the repartitioning time, while the partitioning quality it achieves is on par with that of the best existing static method.
Comments: 21 pages, 15 figures. Under review
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Databases (cs.DB); Discrete Mathematics (cs.DM); Data Structures and Algorithms (cs.DS); Social and Information Networks (cs.SI)
Cite as: arXiv:2101.07026 [cs.DC]
  (or arXiv:2101.07026v1 [cs.DC] for this version)

Submission history

From: Masatoshi Hanai [view email]
[v1] Mon, 18 Jan 2021 12:06:00 GMT (1621kb)

Link back to: arXiv, form interface, contact.