Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach
(Submitted on 13 Sep 2016 (v1), last revised 22 Mar 2017 (this version, v2))
Abstract: We present a theoretically grounded approach to train deep neural networks, including recurrent networks, subject to class-dependent label noise. We propose two procedures for loss correction that are agnostic to both application domain and network architecture. They simply amount to at most a matrix inversion and multiplication, provided that we know the probability of each class being corrupted into another. We further show how one can estimate these probabilities, adapting a recent technique for noise estimation to the multi-class setting, and thus providing an end-to-end framework. Extensive experiments on MNIST, IMDB, CIFAR-10, CIFAR-100 and a large scale dataset of clothing images employing a diversity of architectures --- stacking dense, convolutional, pooling, dropout, batch normalization, word embedding, LSTM and residual layers --- demonstrate the noise robustness of our proposals. Incidentally, we also prove that, when ReLU is the only non-linearity, the loss curvature is immune to class-dependent label noise.
Submission history
From: Giorgio Patrini [view email][v1] Tue, 13 Sep 2016 05:23:29 GMT (380kb,D)
[v2] Wed, 22 Mar 2017 08:48:02 GMT (388kb,D)
Link back to: arXiv, form interface, contact.