Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: LSTM-based Online Learning: An Efficient EKF Based Algorithm with a Convergence Guarantee
(Submitted on 22 Oct 2019 (v1), revised 9 Nov 2019 (this version, v2), latest version 31 May 2021 (v5))
Abstract: We investigate online nonlinear regression with long short term memory (LSTM) based networks, which we refer to as LSTM-based online learning. For LSTM-based online learning, we introduce a highly efficient extended Kalman filter (EKF) based training algorithm with a theoretical convergence guarantee. Our algorithm is truly online such that it does not make any assumption on the underlying data generating process or the system dynamics of the learning algorithm to guarantee convergence. Through an extensive set of simulations, we illustrate significant performance improvements achieved by our algorithm with respect to the conventional LSTM training methods. We particularly show that our algorithm provides very similar error performance with the EKF learning algorithm in 10-40 times shorter training time depending on the parameter size of the network.
Submission history
From: Nuri Mert Vural [view email][v1] Tue, 22 Oct 2019 09:30:41 GMT (70kb,D)
[v2] Sat, 9 Nov 2019 16:40:10 GMT (516kb,D)
[v3] Sat, 7 Mar 2020 16:12:42 GMT (517kb,D)
[v4] Sat, 15 Aug 2020 13:43:54 GMT (158kb,D)
[v5] Mon, 31 May 2021 15:39:14 GMT (1094kb,D)
Link back to: arXiv, form interface, contact.