We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: The Compact Support Neural Network

Abstract: Neural networks are popular and useful in many fields, but they have the problem of giving high confidence responses for examples that are away from the training data. This makes the neural networks very confident in their prediction while making gross mistakes, thus limiting their reliability for safety-critical applications such as autonomous driving, space exploration, etc. In this paper, we present a neuron generalization that has the standard dot-product-based neuron and the RBF neuron as two extreme cases of a shape parameter. Using ReLU as the activation function we obtain a novel neuron that has compact support, which means its output is zero outside a bounded domain. We show how to avoid difficulties in training a neural network with such neurons, by starting with a trained standard neural network and gradually increasing the shape parameter to the desired value. Through experiments on standard benchmark datasets, we show the promise of the proposed approach, in that it can have good prediction accuracy on in-distribution samples while being able to consistently detect and have low confidence on out-of-distribution samples.
Comments: 8 pages, 9 figures
Subjects: Machine Learning (cs.LG)
Cite as: arXiv:2104.00269 [cs.LG]
  (or arXiv:2104.00269v1 [cs.LG] for this version)

Submission history

From: Adrian Barbu [view email]
[v1] Thu, 1 Apr 2021 06:08:09 GMT (661kb,D)
[v2] Sat, 20 Nov 2021 22:23:12 GMT (1075kb,D)

Link back to: arXiv, form interface, contact.