References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Robust Assessment of Real-World Adversarial Examples
(Submitted on 24 Nov 2019 (v1), last revised 15 Mar 2020 (this version, v2))
Abstract: We explore rigorous, systematic, and controlled experimental evaluation of adversarial examples in the real world and propose a testing regimen for evaluation of real world adversarial objects. We show that for small scene/ environmental perturbations, large adversarial performance differences exist. Current state of adversarial reporting exists largely as a frequency count over a dynamic collections of scenes. Our work underscores the need for either a more complete report or a score that incorporates scene changes and baseline performance for models and environments tested by adversarial developers. We put forth a score that attempts to address the above issues in a straight-forward exemplar application for multiple generated adversary examples. We contribute the following: 1. a testbed for adversarial assessment, 2. a score for adversarial examples, and 3. a collection of additional evaluations on testbed data.
Submission history
From: Carlos M. Ortiz [view email][v1] Sun, 24 Nov 2019 00:00:33 GMT (4261kb,D)
[v2] Sun, 15 Mar 2020 01:21:42 GMT (8733kb,D)
Link back to: arXiv, form interface, contact.