We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: OpTorch: Optimized deep learning architectures for resource limited environments

Abstract: Deep learning algorithms have made many breakthroughs and have various applications in real life. Computational resources become a bottleneck as the data and complexity of the deep learning pipeline increases. In this paper, we propose optimized deep learning pipelines in multiple aspects of training including time and memory. OpTorch is a machine learning library designed to overcome weaknesses in existing implementations of neural network training. OpTorch provides features to train complex neural networks with limited computational resources. OpTorch achieved the same accuracy as existing libraries on Cifar-10 and Cifar-100 datasets while reducing memory usage to approximately 50%. We also explore the effect of weights on total memory usage in deep learning pipelines. In our experiments, parallel encoding-decoding along with sequential checkpoints results in much improved memory and time usage while keeping the accuracy similar to existing pipelines. OpTorch python package is available at available at this https URL
Subjects: Machine Learning (cs.LG); Hardware Architecture (cs.AR); Machine Learning (stat.ML)
Cite as: arXiv:2105.00619 [cs.LG]
  (or arXiv:2105.00619v2 [cs.LG] for this version)

Submission history

From: Salman Ahmed [view email]
[v1] Mon, 3 May 2021 03:58:57 GMT (898kb,D)
[v2] Tue, 4 May 2021 09:25:55 GMT (897kb,D)

Link back to: arXiv, form interface, contact.