Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Statistical Characteristics of Deep Representations: An Empirical Investigation
(Submitted on 8 Nov 2018 (v1), last revised 2 Dec 2020 (this version, v2))
Abstract: In this study, the effects of eight representation regularization methods are investigated, including two newly developed rank regularizers (RR). The investigation shows that the statistical characteristics of representations such as correlation, sparsity, and rank can be manipulated as intended, during training. Furthermore, it is possible to improve the baseline performance simply by trying all the representation regularizers and fine-tuning the strength of their effects. In contrast to performance improvement, no consistent relationship between performance and statistical characteristics was observable. The results indicate that manipulation of statistical characteristics can be helpful for improving performance, but only indirectly through its influence on learning dynamics or its tuning effects.
Submission history
From: Daeyoung Choi [view email][v1] Thu, 8 Nov 2018 20:17:50 GMT (1172kb,D)
[v2] Wed, 2 Dec 2020 11:39:32 GMT (8151kb,D)
Link back to: arXiv, form interface, contact.