We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.DC

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Distributed, Parallel, and Cluster Computing

Title: OpTree: An Efficient Algorithm for All-gather Operation in Optical Interconnect Systems

Abstract: All-gather collective communication is one of the most important communication primitives in parallel and distributed computation, which plays an essential role in many HPC applications such as distributed Deep Learning (DL) with model and hybrid parallelism. To solve the communication bottleneck of All-gather, optical interconnection network can provide unprecedented high bandwidth and reliability for data transfer among the distributed nodes. However, most traditional All-gather algorithms are designed for electrical interconnection, which cannot fit well for optical interconnect systems, resulting in poor performance. This paper proposes an efficient scheme, called OpTree, for All-gather operation on optical interconnect systems. OpTree derives an optimal $m$-ary tree corresponding to the optimal number of communication stages, achieving minimum communication time. We further analyze and compare the communication steps of OpTree with existing All-gather algorithms. Theoretical results exhibit that OpTree requires much less number of communication steps than existing All-gather algorithms on optical interconnect systems. Simulation results show that OpTree can reduce communication time by 72.21%, 94.30%, and 88.58%, respectively, compared with three existing All-gather schemes, WRHT, Ring, and NE.
Comments: This paper is under review at a conference
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC)
Cite as: arXiv:2211.15119 [cs.DC]
  (or arXiv:2211.15119v1 [cs.DC] for this version)

Submission history

From: Travis(Fei) Dai [view email]
[v1] Mon, 28 Nov 2022 08:17:21 GMT (2866kb,D)

Link back to: arXiv, form interface, contact.