We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Generalization Error Bounds on Deep Learning with Markov Datasets

Authors: Lan V. Truong
Abstract: In this paper, we derive upper bounds on generalization errors for deep neural networks with Markov datasets. These bounds are developed based on Koltchinskii and Panchenko's approach for bounding the generalization error of combined classifiers with i.i.d. datasets. The development of new symmetrization inequalities in high-dimensional probability for Markov chains is a key element in our extension, where the spectral gap of the infinitesimal generator of the Markov chain plays a key parameter in these inequalities. We also propose a simple method to convert these bounds and other similar ones in traditional deep learning and machine learning to Bayesian counterparts for both i.i.d. and Markov datasets. Extensions to $m$-order homogeneous Markov chains such as AR and ARMA models and mixtures of several Markov data services are given.
Comments: Corrected a mistake in proof. Under review for possible publication
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Probability (math.PR); Statistics Theory (math.ST)
Cite as: arXiv:2201.11059 [stat.ML]
  (or arXiv:2201.11059v3 [stat.ML] for this version)

Submission history

From: Lan Truong [view email]
[v1] Thu, 23 Dec 2021 11:14:43 GMT (34kb)
[v2] Thu, 27 Jan 2022 21:46:27 GMT (44kb)
[v3] Mon, 16 May 2022 06:56:42 GMT (47kb)

Link back to: arXiv, form interface, contact.