We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Interpolation and approximation via Momentum ResNets and Neural ODEs

Abstract: In this article, we explore the effects of memory terms in continuous-layer Deep Residual Networks by studying Neural ODEs (NODEs). We investigate two types of models. On one side, we consider the case of Residual Neural Networks with dependence on multiple layers, more precisely Momentum ResNets. On the other side, we analyze a Neural ODE with auxiliary states playing the role of memory states. We examine the interpolation and universal approximation properties for both architectures through a simultaneous control perspective. We also prove the ability of the second model to represent sophisticated maps, such as parametrizations of time-dependent functions. Numerical simulations complement our study.
Subjects: Optimization and Control (math.OC)
MSC classes: 34H05, 37N35, 93B05
Cite as: arXiv:2110.08761 [math.OC]
  (or arXiv:2110.08761v1 [math.OC] for this version)

Submission history

From: Domènec Ruiz-Balet [view email]
[v1] Sun, 17 Oct 2021 08:49:12 GMT (4829kb,D)

Link back to: arXiv, form interface, contact.