References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: GradAug: A New Regularization Method for Deep Neural Networks
(Submitted on 14 Jun 2020 (v1), last revised 12 Oct 2020 (this version, v2))
Abstract: We propose a new regularization method to alleviate over-fitting in deep neural networks. The key idea is utilizing randomly transformed training samples to regularize a set of sub-networks, which are originated by sampling the width of the original network, in the training process. As such, the proposed method introduces self-guided disturbances to the raw gradients of the network and therefore is termed as Gradient Augmentation (GradAug). We demonstrate that GradAug can help the network learn well-generalized and more diverse representations. Moreover, it is easy to implement and can be applied to various structures and applications. GradAug improves ResNet-50 to 78.79% on ImageNet classification, which is a new state-of-the-art accuracy. By combining with CutMix, it further boosts the performance to 79.67%, which outperforms an ensemble of advanced training tricks. The generalization ability is evaluated on COCO object detection and instance segmentation where GradAug significantly surpasses other state-of-the-art methods. GradAug is also robust to image distortions and FGSM adversarial attacks and is highly effective in low data regimes. Code is available at this https URL
Submission history
From: Taojiannan Yang [view email][v1] Sun, 14 Jun 2020 19:30:34 GMT (2102kb,D)
[v2] Mon, 12 Oct 2020 18:20:51 GMT (2125kb,D)
Link back to: arXiv, form interface, contact.