Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Evaluating Robustness of Predictive Uncertainty Estimation: Are Dirichlet-based Models Reliable?
(Submitted on 28 Oct 2020 (v1), last revised 11 Jun 2021 (this version, v2))
Abstract: Dirichlet-based uncertainty (DBU) models are a recent and promising class of uncertainty-aware models. DBU models predict the parameters of a Dirichlet distribution to provide fast, high-quality uncertainty estimates alongside with class predictions. In this work, we present the first large-scale, in-depth study of the robustness of DBU models under adversarial attacks. Our results suggest that uncertainty estimates of DBU models are not robust w.r.t. three important tasks: (1) indicating correctly and wrongly classified samples; (2) detecting adversarial examples; and (3) distinguishing between in-distribution (ID) and out-of-distribution (OOD) data. Additionally, we explore the first approaches to make DBU models more robust. While adversarial training has a minor effect, our median smoothing based approach significantly increases robustness of DBU models.
Submission history
From: Anna-Kathrin Kopetzki [view email][v1] Wed, 28 Oct 2020 13:53:43 GMT (813kb,D)
[v2] Fri, 11 Jun 2021 09:32:11 GMT (1257kb,D)
Link back to: arXiv, form interface, contact.