We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Neural Network Architecture Optimization through Submodularity and Supermodularity

Abstract: Deep learning models' architectures, including depth and width, are key factors influencing models' performance, such as test accuracy and computation time. This paper solves two problems: given computation time budget, choose an architecture to maximize accuracy, and given accuracy requirement, choose an architecture to minimize computation time. We convert this architecture optimization into a subset selection problem. With accuracy's submodularity and computation time's supermodularity, we propose efficient greedy optimization algorithms. The experiments demonstrate our algorithm's ability to find more accurate models or faster models. By analyzing architecture evolution with growing time budget, we discuss relationships among accuracy, time and architecture, and give suggestions on neural network architecture design.
Comments: Withdrawn due to incompleteness and some overlaps with existing literatures, I will resubmit adding further results
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Cite as: arXiv:1609.00074 [stat.ML]
  (or arXiv:1609.00074v3 [stat.ML] for this version)

Submission history

From: Junqi Jin [view email]
[v1] Thu, 1 Sep 2016 00:59:30 GMT (3941kb,D)
[v2] Sun, 19 Mar 2017 13:23:34 GMT (3941kb,D)
[v3] Wed, 21 Feb 2018 03:45:19 GMT (0kb,I)

Link back to: arXiv, form interface, contact.