We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Convergence of Deep Convolutional Neural Networks

Abstract: Convergence of deep neural networks as the depth of the networks tends to infinity is fundamental in building the mathematical foundation for deep learning. In a previous study, we investigated this question for deep ReLU networks with a fixed width. This does not cover the important convolutional neural networks where the widths are increasing from layer to layer. For this reason, we first study convergence of general ReLU networks with increasing widths and then apply the results obtained to deep convolutional neural networks. It turns out the convergence reduces to convergence of infinite products of matrices with increasing sizes, which has not been considered in the literature. We establish sufficient conditions for convergence of such infinite products of matrices. Based on the conditions, we present sufficient conditions for piecewise convergence of general deep ReLU networks with increasing widths, and as well as pointwise convergence of deep ReLU convolutional neural networks.
Comments: arXiv admin note: text overlap with arXiv:2107.12530
Subjects: Machine Learning (cs.LG)
Cite as: arXiv:2109.13542 [cs.LG]
  (or arXiv:2109.13542v2 [cs.LG] for this version)

Submission history

From: Haizhang Zhang [view email]
[v1] Tue, 28 Sep 2021 07:48:17 GMT (16kb)
[v2] Sun, 23 Jan 2022 07:36:47 GMT (17kb)

Link back to: arXiv, form interface, contact.