Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Post Training in Deep Learning with Last Kernel
(Submitted on 14 Nov 2016 (v1), last revised 31 Oct 2017 (this version, v2))
Abstract: One of the main challenges of deep learning methods is the choice of an appropriate training strategy. In particular, additional steps, such as unsupervised pre-training, have been shown to greatly improve the performances of deep structures. In this article, we propose an extra training step, called post-training, which only optimizes the last layer of the network. We show that this procedure can be analyzed in the context of kernel theory, with the first layers computing an embedding of the data and the last layer a statistical model to solve the task based on this embedding. This step makes sure that the embedding, or representation, of the data is used in the best possible way for the considered task. This idea is then tested on multiple architectures with various data sets, showing that it consistently provides a boost in performance.
Submission history
From: Thomas Moreau [view email][v1] Mon, 14 Nov 2016 17:54:28 GMT (282kb,D)
[v2] Tue, 31 Oct 2017 09:25:13 GMT (71kb,D)
Link back to: arXiv, form interface, contact.