Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models
(Submitted on 1 Jul 2021 (v1), last revised 16 Oct 2021 (this version, v2))
Abstract: Supervised learning models often make systematic errors on rare subsets of the data. When these subsets correspond to explicit labels in the data (e.g., gender, race) such poor performance can be identified straightforwardly. This paper introduces a method for discovering systematic errors that do not correspond to such explicitly labelled subgroups. The key idea is that similar inputs tend to have similar representations in the final hidden layer of a neural network. We leverage this structure by "shining a spotlight" on this representation space to find contiguous regions where the model performs poorly. We show that the spotlight surfaces semantically meaningful areas of weakness in a wide variety of existing models spanning computer vision, NLP, and recommender systems.
Submission history
From: Greg d'Eon [view email][v1] Thu, 1 Jul 2021 21:58:00 GMT (4599kb,D)
[v2] Sat, 16 Oct 2021 03:58:42 GMT (5622kb,D)
Link back to: arXiv, form interface, contact.