Current browse context:
cs.CY
Change to browse by:
References & Citations
Computer Science > Computers and Society
Title: Lucid Explanations Help: Using a Human-AI Image-Guessing Game to Evaluate Machine Explanation Helpfulness
(Submitted on 5 Apr 2019 (v1), revised 9 Apr 2019 (this version, v2), latest version 21 Sep 2019 (v4))
Abstract: While there have been many proposals on how to make AI algorithms more transparent, few have attempted to evaluate the impact of AI explanations on human performance on a task using AI. We propose a Twenty-Questions style collaborative image guessing game, Explanation-assisted Guess Which (ExAG) as a method of evaluating the efficacy of explanations in the context of Visual Question Answering (VQA) - the task of answering natural language questions on images. We study the effect of VQA agent explanations on the game performance as a function of explanation type and quality. We observe that "helpful" explanations are conducive to game performance (by almost 22% for "excellent" rated explanation games), and having at least one "correct" explanation is significantly helpful when VQA system answers are mostly noisy (by almost 30% compared to no explanation games). We also see that players develop a preference for explanations even when penalized and that the explanations are mostly rated as "helpful".
Submission history
From: Arijit Ray [view email][v1] Fri, 5 Apr 2019 21:26:39 GMT (6538kb,D)
[v2] Tue, 9 Apr 2019 16:45:38 GMT (6574kb,D)
[v3] Tue, 2 Jul 2019 21:52:10 GMT (6233kb,D)
[v4] Sat, 21 Sep 2019 17:13:50 GMT (4998kb,D)
Link back to: arXiv, form interface, contact.