We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.AR

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Hardware Architecture

Title: TaxoNN: A Light-Weight Accelerator for Deep Neural Network Training

Abstract: Emerging intelligent embedded devices rely on Deep Neural Networks (DNNs) to be able to interact with the real-world environment. This interaction comes with the ability to retrain DNNs, since environmental conditions change continuously in time. Stochastic Gradient Descent (SGD) is a widely used algorithm to train DNNs by optimizing the parameters over the training data iteratively. In this work, first we present a novel approach to add the training ability to a baseline DNN accelerator (inference only) by splitting the SGD algorithm into simple computational elements. Then, based on this heuristic approach we propose TaxoNN, a light-weight accelerator for DNN training. TaxoNN can easily tune the DNN weights by reusing the hardware resources used in the inference process using a time-multiplexing approach and low-bitwidth units. Our experimental results show that TaxoNN delivers, on average, 0.97% higher misclassification rate compared to a full-precision implementation. Moreover, TaxoNN provides 2.1$\times$ power saving and 1.65$\times$ area reduction over the state-of-the-art DNN training accelerator.
Comments: Accepted to ISCAS 2020. 5 pages, 5 figures
Subjects: Hardware Architecture (cs.AR); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
Journal reference: 2020 IEEE International Symposium on Circuits and Systems (ISCAS), 2020, pp. 1-5
DOI: 10.1109/ISCAS45731.2020.9181001
Cite as: arXiv:2010.05197 [cs.AR]
  (or arXiv:2010.05197v1 [cs.AR] for this version)

Submission history

From: Kossar Pourahmadi Meibodi [view email]
[v1] Sun, 11 Oct 2020 09:04:19 GMT (4499kb,D)

Link back to: arXiv, form interface, contact.