Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Universality and approximation bounds for echo state networks with random weights
(Submitted on 12 Jun 2022 (v1), last revised 4 Dec 2023 (this version, v3))
Abstract: We study the uniform approximation of echo state networks with randomly generated internal weights. These models, in which only the readout weights are optimized during training, have made empirical success in learning dynamical systems. Recent results showed that echo state networks with ReLU activation are universal. In this paper, we give an alternative construction and prove that the universality holds for general activation functions. Specifically, our main result shows that, under certain condition on the activation function, there exists a sampling procedure for the internal weights so that the echo state network can approximate any continuous casual time-invariant operators with high probability. In particular, for ReLU activation, we give explicit construction for these sampling procedures. We also quantify the approximation error of the constructed ReLU echo state networks for sufficiently regular operators.
Submission history
From: Yunfei Yang [view email][v1] Sun, 12 Jun 2022 05:43:16 GMT (41kb,D)
[v2] Tue, 11 Oct 2022 09:29:16 GMT (42kb,D)
[v3] Mon, 4 Dec 2023 02:05:25 GMT (38kb)
Link back to: arXiv, form interface, contact.