We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.DC

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Computer Science > Distributed, Parallel, and Cluster Computing

Title: The Deep Learning Compiler: A Comprehensive Survey

Abstract: The difficulty of deploying various deep learning (DL) models on diverse DL hardware has boosted the research and development of DL compilers in the community. Several DL compilers have been proposed from both industry and academia such as Tensorflow XLA and TVM. Similarly, the DL compilers take the DL models described in different DL frameworks as input, and then generate optimized codes for diverse DL hardware as output. However, none of the existing survey has analyzed the unique design architecture of the DL compilers comprehensively. In this paper, we perform a comprehensive survey of existing DL compilers by dissecting the commonly adopted design in details, with emphasis on the DL oriented multi-level IRs, and frontend/backend optimizations. Specifically, we provide a comprehensive comparison among existing DL compilers from various aspects. In addition, we present detailed analysis on the design of multi-level IRs and illustrate the commonly adopted optimization techniques. Finally, several insights are highlighted as the potential research directions of DL compiler. This is the first survey paper focusing on the design architecture of DL compilers, which we hope can pave the road for future research towards DL compiler.
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Machine Learning (cs.LG); Performance (cs.PF)
Journal reference: IEEE Transactions on Parallel & Distributed Systems, vol. 32, no. 03, pp. 708-727, 2021
DOI: 10.1109/TPDS.2020.3030548
Cite as: arXiv:2002.03794 [cs.DC]
  (or arXiv:2002.03794v4 [cs.DC] for this version)

Submission history

From: Mingzhen Li [view email]
[v1] Thu, 6 Feb 2020 07:29:08 GMT (680kb,D)
[v2] Thu, 27 Feb 2020 05:08:44 GMT (672kb,D)
[v3] Sun, 5 Apr 2020 10:34:01 GMT (481kb,D)
[v4] Fri, 28 Aug 2020 09:19:43 GMT (568kb,D)

Link back to: arXiv, form interface, contact.