We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Diverse Gaussian Noise Consistency Regularization for Robustness and Uncertainty Calibration

Abstract: Deep neural networks achieve high prediction accuracy when the train and test distributions coincide. In practice though, various types of corruptions occur which deviate from this setup and cause severe performance degradations. Few methods have been proposed to address generalization in the presence of unforeseen domain shifts. In particular, digital noise corruptions arise commonly in practice during the image acquisition stage and present a significant challenge for current robustness approaches. In this paper, we propose a diverse Gaussian noise consistency regularization method for improving robustness of image classifiers under a variety of noise corruptions while still maintaining high clean accuracy. We derive bounds to motivate and understand the behavior of our Gaussian noise consistency regularization using a local loss landscape analysis. We show that this simple approach improves robustness against various unforeseen noise corruptions by 4.2-18.4% over adversarial training and other strong diverse data augmentation baselines across several benchmarks. Furthermore, when combined with state-of-the-art diverse data augmentation techniques, experiments against state-of-the-art show our method further improves robustness accuracy by 3.7% and uncertainty calibration by 5.5% for all common corruptions on several image classification benchmarks.
Comments: Under review. Preliminary version accepted to ICML 2021 Uncertainty & Robustness in Deep Learning Workshop
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
Cite as: arXiv:2104.01231 [cs.LG]
  (or arXiv:2104.01231v5 [cs.LG] for this version)

Submission history

From: Theodoros Tsiligkaridis [view email]
[v1] Fri, 2 Apr 2021 20:25:53 GMT (579kb,D)
[v2] Sat, 25 Sep 2021 20:13:53 GMT (811kb,D)
[v3] Sat, 22 Jan 2022 13:27:58 GMT (473kb,D)
[v4] Sat, 21 May 2022 20:31:11 GMT (526kb,D)
[v5] Tue, 29 Nov 2022 15:32:46 GMT (549kb,D)

Link back to: arXiv, form interface, contact.