Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Geometry Perspective Of Estimating Learning Capability Of Neural Networks
(Submitted on 3 Nov 2020 (v1), last revised 1 Dec 2020 (this version, v2))
Abstract: The paper uses statistical and differential geometric motivation to acquire prior information about the learning capability of an artificial neural network on a given dataset. The paper considers a broad class of neural networks with generalized architecture performing simple least square regression with stochastic gradient descent (SGD). The system characteristics at two critical epochs in the learning trajectory are analyzed. During some epochs of the training phase, the system reaches equilibrium with the generalization capability attaining a maximum. The system can also be coherent with localized, non-equilibrium states, which is characterized by the stabilization of the Hessian matrix. The paper proves that neural networks with higher generalization capability will have a slower convergence rate. The relationship between the generalization capability with the stability of the neural network has also been discussed. By correlating the principles of high-energy physics with the learning theory of neural networks, the paper establishes a variant of the Complexity-Action conjecture from an artificial neural network perspective.
Submission history
From: Ankan Dutta [view email][v1] Tue, 3 Nov 2020 12:03:19 GMT (31kb)
[v2] Tue, 1 Dec 2020 07:32:38 GMT (28kb,D)
Link back to: arXiv, form interface, contact.