Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Parametric Exponential Linear Unit for Deep Convolutional Neural Networks
(Submitted on 30 May 2016 (v1), last revised 10 Jan 2018 (this version, v4))
Abstract: Object recognition is an important task for improving the ability of visual systems to perform complex scene understanding. Recently, the Exponential Linear Unit (ELU) has been proposed as a key component for managing bias shift in Convolutional Neural Networks (CNNs), but defines a parameter that must be set by hand. In this paper, we propose learning a parameterization of ELU in order to learn the proper activation shape at each layer in the CNNs. Our results on the MNIST, CIFAR-10/100 and ImageNet datasets using the NiN, Overfeat, All-CNN and ResNet networks indicate that our proposed Parametric ELU (PELU) has better performances than the non-parametric ELU. We have observed as much as a 7.28% relative error improvement on ImageNet with the NiN network, with only 0.0003% parameter increase. Our visual examination of the non-linear behaviors adopted by Vgg using PELU shows that the network took advantage of the added flexibility by learning different activations at different layers.
Submission history
From: Ludovic Trottier [view email][v1] Mon, 30 May 2016 17:16:40 GMT (78kb)
[v2] Tue, 31 May 2016 19:24:04 GMT (75kb)
[v3] Fri, 18 Nov 2016 20:26:25 GMT (208kb,D)
[v4] Wed, 10 Jan 2018 15:18:48 GMT (1846kb)
Link back to: arXiv, form interface, contact.