Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: The empirical size of trained neural networks
(Submitted on 29 Nov 2016)
Abstract: ReLU neural networks define piecewise linear functions of their inputs. However, initializing and training a neural network is very different from fitting a linear spline. In this paper, we expand empirically upon previous theoretical work to demonstrate features of trained neural networks. Standard network initialization and training produce networks vastly simpler than a naive parameter count would suggest and can impart odd features to the trained network. However, we also show the forced simplicity is beneficial and, indeed, critical for the wide success of these networks.
Link back to: arXiv, form interface, contact.