We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Functional Analysis

Title: Approximation Theory of Tree Tensor Networks: Tensorized Multivariate Functions

Abstract: We study the approximation of multivariate functions with tensor networks (TNs). The main conclusion of this work is an answer to the following two questions: "What are the approximation capabilities of TNs?" and "What is an appropriate model class of functions that can be approximated with TNs?" To answer the former: we show that TNs can (near to) optimally replicate $h$-uniform and $h$-adaptive approximation, for any smoothness order of the target function. Tensor networks thus exhibit universal expressivity w.r.t. isotropic, anisotropic and mixed smoothness spaces that is comparable with more general neural networks families such as deep rectified linear unit (ReLU) networks. Put differently, TNs have the capacity to (near to) optimally approximate many function classes -- without being adapted to the particular class in question. To answer the latter: as a candidate model class we consider approximation classes of TNs and show that these are (quasi-)Banach spaces, that many types of classical smoothness spaces are continuously embedded into said approximation classes and that TN approximation classes are themselves not embedded in any classical smoothness space.
Comments: For part I see arXiv:2007.00118, for part II see arXiv:2007.00128
Subjects: Functional Analysis (math.FA); Machine Learning (cs.LG); Numerical Analysis (math.NA)
MSC classes: 41A65, 41A15, 41A10 (primary), 68T05, 42C40, 65D99 (secondary)
Cite as: arXiv:2101.11932 [math.FA]
  (or arXiv:2101.11932v3 [math.FA] for this version)

Submission history

From: Mazen Ali [view email]
[v1] Thu, 28 Jan 2021 11:09:40 GMT (150kb,D)
[v2] Sun, 15 Jan 2023 18:43:15 GMT (164kb,D)
[v3] Wed, 18 Jan 2023 15:17:54 GMT (963kb,D)

Link back to: arXiv, form interface, contact.