Current browse context:
stat
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Towards Robust Neural Networks via Random Self-ensemble
(Submitted on 2 Dec 2017 (v1), last revised 1 Aug 2018 (this version, v2))
Abstract: Recent studies have revealed the vulnerability of deep neural networks: A small adversarial perturbation that is imperceptible to human can easily make a well-trained deep neural network misclassify. This makes it unsafe to apply neural networks in security-critical applications. In this paper, we propose a new defense algorithm called Random Self-Ensemble (RSE) by combining two important concepts: {\bf randomness} and {\bf ensemble}. To protect a targeted model, RSE adds random noise layers to the neural network to prevent the strong gradient-based attacks, and ensembles the prediction over random noises to stabilize the performance. We show that our algorithm is equivalent to ensemble an infinite number of noisy models $f_\epsilon$ without any additional memory overhead, and the proposed training procedure based on noisy stochastic gradient descent can ensure the ensemble model has a good predictive capability. Our algorithm significantly outperforms previous defense techniques on real data sets. For instance, on CIFAR-10 with VGG network (which has 92\% accuracy without any attack), under the strong C\&W attack within a certain distortion tolerance, the accuracy of unprotected model drops to less than 10\%, the best previous defense technique has $48\%$ accuracy, while our method still has $86\%$ prediction accuracy under the same level of attack. Finally, our method is simple and easy to integrate into any neural network.
Submission history
From: Xuanqing Liu [view email][v1] Sat, 2 Dec 2017 22:26:12 GMT (605kb,D)
[v2] Wed, 1 Aug 2018 00:44:31 GMT (255kb,D)
Link back to: arXiv, form interface, contact.