Current browse context:
cs.CV
Change to browse by:
References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Weight and Gradient Centralization in Deep Neural Networks
(Submitted on 2 Oct 2020 (v1), last revised 17 Jan 2021 (this version, v3))
Abstract: Batch normalization is currently the most widely used variant of internal normalization for deep neural networks. Additional work has shown that the normalization of weights and additional conditioning as well as the normalization of gradients further improve the generalization. In this work, we combine several of these methods and thereby increase the generalization of the networks. The advantage of the newer methods compared to the batch normalization is not only increased generalization, but also that these methods only have to be applied during training and, therefore, do not influence the running time during use. Link to CUDA code this https URL
Submission history
From: Wolfgang Fuhl [view email][v1] Fri, 2 Oct 2020 08:50:04 GMT (803kb,D)
[v2] Fri, 30 Oct 2020 14:13:39 GMT (803kb,D)
[v3] Sun, 17 Jan 2021 12:05:14 GMT (277kb,D)
Link back to: arXiv, form interface, contact.