Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning
(Submitted on 1 Jan 2020 (v1), last revised 22 Jan 2020 (this version, v4))
Abstract: With the emergence of a spectrum of high-end mobile devices, many applications that formerly required desktop-level computation capability are being transferred to these devices. However, executing the inference of Deep Neural Networks (DNNs) is still challenging considering high computation and storage demands, specifically, if real-time performance with high accuracy is needed. Weight pruning of DNNs is proposed, but existing schemes represent two extremes in the design space: non-structured pruning is fine-grained, accurate, but not hardware friendly; structured pruning is coarse-grained, hardware-efficient, but with higher accuracy loss. In this paper, we introduce a new dimension, fine-grained pruning patterns inside the coarse-grained structures, revealing a previously unknown point in design space. With the higher accuracy enabled by fine-grained pruning patterns, the unique insight is to use the compiler to re-gain and guarantee high hardware efficiency. In other words, our method achieves the best of both worlds, and is desirable across theory/algorithm, compiler, and hardware levels. The proposed PatDNN is an end-to-end framework to efficiently execute DNN on mobile devices with the help of a novel model compression technique (pattern-based pruning based on extended ADMM solution framework) and a set of thorough architecture-aware compiler- and code generation-based optimizations (filter kernel reordering, compressed weight storage, register load redundancy elimination, and parameter auto-tuning). Evaluation results demonstrate that PatDNN outperforms three state-of-the-art end-to-end DNN frameworks, TensorFlow Lite, TVM, and Alibaba Mobile Neural Network with speedup up to 44.5x, 11.4x, and 7.1x, respectively, with no accuracy compromise. Real-time inference of representative large-scale DNNs (e.g., VGG-16, ResNet-50) can be achieved using mobile devices.
Submission history
From: Xiaolong Ma [view email][v1] Wed, 1 Jan 2020 04:52:07 GMT (5321kb,D)
[v2] Fri, 10 Jan 2020 00:27:57 GMT (5325kb,D)
[v3] Fri, 17 Jan 2020 04:32:38 GMT (5315kb,D)
[v4] Wed, 22 Jan 2020 04:13:06 GMT (5419kb,D)
Link back to: arXiv, form interface, contact.