Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Lifted Neural Networks
(Submitted on 3 May 2018 (v1), last revised 21 Jun 2018 (this version, v2))
Abstract: We describe a novel family of models of multi- layer feedforward neural networks in which the activation functions are encoded via penalties in the training problem. Our approach is based on representing a non-decreasing activation function as the argmin of an appropriate convex optimiza- tion problem. The new framework allows for algo- rithms such as block-coordinate descent methods to be applied, in which each step is composed of a simple (no hidden layer) supervised learning problem that is parallelizable across data points and/or layers. Experiments indicate that the pro- posed models provide excellent initial guesses for weights for standard neural networks. In addi- tion, the model provides avenues for interesting extensions, such as robustness against noisy in- puts and optimizing over parameters in activation functions.
Submission history
From: Geoffrey Negiar [view email][v1] Thu, 3 May 2018 20:56:31 GMT (870kb,D)
[v2] Thu, 21 Jun 2018 01:48:34 GMT (905kb,D)
Link back to: arXiv, form interface, contact.