We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Realization Theory Of Recurrent Neural ODEs Using Polynomial System Embeddings

Abstract: In this paper we show that neural ODE analogs of recurrent (ODE-RNN) and Long Short-Term Memory (ODE-LSTM) networks can be algorithmically embeddeded into the class of polynomial systems. This embedding preserves input-output behavior and can suitably be extended to other neural DE architectures. We then use realization theory of polynomial systems to provide necessary conditions for an input-output map to be realizable by an ODE-LSTM and sufficient conditions for minimality of such systems. These results represent the first steps towards realization theory of recurrent neural ODE architectures, which is is expected be useful for model reduction and learning algorithm analysis of recurrent neural ODEs.
Comments: 10 pages. Comments are welcome!
Subjects: Optimization and Control (math.OC); Machine Learning (cs.LG)
Cite as: arXiv:2205.11989 [math.OC]
  (or arXiv:2205.11989v1 [math.OC] for this version)

Submission history

From: Martin Gonzalez [view email]
[v1] Tue, 24 May 2022 11:36:18 GMT (21kb)

Link back to: arXiv, form interface, contact.