We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Computer Science > Machine Learning

Title: Liquid Time-constant Networks

Abstract: We introduce a new class of time-continuous recurrent neural network models. Instead of declaring a learning system's dynamics by implicit nonlinearities, we construct networks of linear first-order dynamical systems modulated via nonlinear interlinked gates. The resulting models represent dynamical systems with varying (i.e., liquid) time-constants coupled to their hidden state, with outputs being computed by numerical differential equation solvers. These neural networks exhibit stable and bounded behavior, yield superior expressivity within the family of neural ordinary differential equations, and give rise to improved performance on time-series prediction tasks. To demonstrate these properties, we first take a theoretical approach to find bounds over their dynamics and compute their expressive power by the trajectory length measure in latent trajectory space. We then conduct a series of time-series prediction experiments to manifest the approximation capability of Liquid Time-Constant Networks (LTCs) compared to classical and modern RNNs. Code and data are available at this https URL
Comments: Accepted to the Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI-21)
Subjects: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Machine Learning (stat.ML)
Cite as: arXiv:2006.04439 [cs.LG]
  (or arXiv:2006.04439v4 [cs.LG] for this version)

Submission history

From: Ramin Hasani [view email]
[v1] Mon, 8 Jun 2020 09:53:35 GMT (5872kb,D)
[v2] Sat, 12 Sep 2020 18:13:30 GMT (5872kb,D)
[v3] Fri, 20 Nov 2020 01:26:16 GMT (5872kb,D)
[v4] Mon, 14 Dec 2020 22:23:52 GMT (5852kb,D)

Link back to: arXiv, form interface, contact.