Current browse context:
cs
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Distinction Maximization Loss: Efficiently Improving Out-of-Distribution Detection and Uncertainty Estimation by Replacing the Loss and Calibrating
(Submitted on 12 May 2022 (v1), last revised 5 Aug 2022 (this version, v5))
Abstract: Building robust deterministic neural networks remains a challenge. On the one hand, some approaches improve out-of-distribution detection at the cost of reducing classification accuracy in some situations. On the other hand, some methods simultaneously increase classification accuracy, uncertainty estimation, and out-of-distribution detection at the expense of reducing the inference efficiency. In this paper, we propose training deterministic neural networks using our DisMax loss, which works as a drop-in replacement for the usual SoftMax loss (i.e., the combination of the linear output layer, the SoftMax activation, and the cross-entropy loss). Starting from the IsoMax+ loss, we create each logit based on the distances to all prototypes, rather than just the one associated with the correct class. We also introduce a mechanism to combine images to construct what we call fractional probability regularization. Moreover, we present a fast way to calibrate the network after training. Finally, we propose a composite score to perform out-of-distribution detection. Our experiments show that DisMax usually outperforms current approaches simultaneously in classification accuracy, uncertainty estimation, and out-of-distribution detection while maintaining deterministic neural network inference efficiency. The code to reproduce the results is available at this https URL
Submission history
From: David Macêdo [view email][v1] Thu, 12 May 2022 04:37:35 GMT (2039kb,D)
[v2] Thu, 19 May 2022 12:04:27 GMT (2039kb,D)
[v3] Thu, 28 Jul 2022 12:53:20 GMT (2044kb,D)
[v4] Mon, 1 Aug 2022 05:04:23 GMT (2044kb,D)
[v5] Fri, 5 Aug 2022 18:26:34 GMT (2043kb,D)
Link back to: arXiv, form interface, contact.