We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.AI

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Artificial Intelligence

Title: To Trust or Not to Trust a Regressor: Estimating and Explaining Trustworthiness of Regression Predictions

Abstract: In hybrid human-AI systems, users need to decide whether or not to trust an algorithmic prediction while the true error in the prediction is unknown. To accommodate such settings, we introduce RETRO-VIZ, a method for (i) estimating and (ii) explaining trustworthiness of regression predictions. It consists of RETRO, a quantitative estimate of the trustworthiness of a prediction, and VIZ, a visual explanation that helps users identify the reasons for the (lack of) trustworthiness of a prediction. We find that RETRO-scores negatively correlate with prediction error across 117 experimental settings, indicating that RETRO provides a useful measure to distinguish trustworthy predictions from untrustworthy ones. In a user study with 41 participants, we find that VIZ-explanations help users identify whether a prediction is trustworthy or not: on average, 95.1% of participants correctly select the more trustworthy prediction, given a pair of predictions. In addition, an average of 75.6% of participants can accurately describe why a prediction seems to be (not) trustworthy. Finally, we find that the vast majority of users subjectively experience RETRO-VIZ as a useful tool to assess the trustworthiness of algorithmic predictions.
Comments: Accepted to ICML 2021 Workshop on Human in the Loop Learning (HILL)
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2104.06982 [cs.AI]
  (or arXiv:2104.06982v2 [cs.AI] for this version)

Submission history

From: Kim de Bie [view email]
[v1] Wed, 14 Apr 2021 17:04:20 GMT (4744kb,D)
[v2] Wed, 28 Jul 2021 13:29:09 GMT (608kb,D)

Link back to: arXiv, form interface, contact.