We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: GIST: Distributed Training for Large-Scale Graph Convolutional Networks

Abstract: The graph convolutional network (GCN) is a go-to solution for machine learning on graphs, but its training is notoriously difficult to scale in terms of both the size of the graph and the number of model parameters. These limitations are in stark contrast to the increasing scale (in data size and model size) of experiments in deep learning research. In this work, we propose GIST, a novel distributed approach that enables efficient training of wide (overparameterized) GCNs on large graphs. GIST is a hybrid layer and graph sampling method, which disjointly partitions the global model into several, smaller sub-GCNs that are independently trained across multiple GPUs in parallel. This distributed framework improves model performance and significantly decreases wall-clock training time. GIST seeks to enable large-scale GCN experimentation with the goal of bridging the existing gap in scale between graph machine learning and deep learning.
Comments: 17 pages, 5 figures, pre-print under review
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC); Optimization and Control (math.OC)
ACM classes: I.2.4
Cite as: arXiv:2102.10424 [cs.LG]
  (or arXiv:2102.10424v1 [cs.LG] for this version)

Submission history

From: Cameron R. Wolfe [view email]
[v1] Sat, 20 Feb 2021 19:25:38 GMT (3105kb,D)
[v2] Wed, 9 Jun 2021 21:11:07 GMT (149kb,D)
[v3] Thu, 1 Jul 2021 13:14:57 GMT (3087kb,D)
[v4] Mon, 14 Mar 2022 14:12:18 GMT (320kb,D)

Link back to: arXiv, form interface, contact.