We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.ST

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Statistics Theory

Title: On the Convergence of the EM Algorithm: A Data-Adaptive Analysis

Abstract: The Expectation-Maximization (EM) algorithm is an iterative method to maximize the log-likelihood function for parameter estimation. Previous works on the convergence analysis of the EM algorithm have established results on the asymptotic (population level) convergence rate of the algorithm. In this paper, we give a data-adaptive analysis of the sample level local convergence rate of the EM algorithm. In particular, we show that the local convergence rate of the EM algorithm is a random variable $\overline{K}_{n}$ derived from the data generating distribution, which adaptively yields the convergence rate of the EM algorithm on each finite sample data set from the same population distribution. We then give a non-asymptotic concentration bound of $\overline{K}_{n}$ on the population level optimal convergence rate $\overline{\kappa}$ of the EM algorithm, which implies that $\overline{K}_{n}\to\overline{\kappa}$ in probability as the sample size $n\to\infty$. Our theory identifies the effect of sample size on the convergence behavior of sample EM sequence, and explains a surprising phenomenon in applications of the EM algorithm, i.e. the finite sample version of the algorithm sometimes converges faster even than the population version. We apply our theory to the EM algorithm on three canonical models and obtain specific forms of the adaptive convergence theorem for each model.
Comments: 55 pages, 2 figures
Subjects: Statistics Theory (math.ST)
Cite as: arXiv:1611.00519 [math.ST]
  (or arXiv:1611.00519v2 [math.ST] for this version)

Submission history

From: Chong Wu [view email]
[v1] Wed, 2 Nov 2016 09:35:30 GMT (132kb,D)
[v2] Tue, 30 May 2017 14:30:30 GMT (136kb,D)

Link back to: arXiv, form interface, contact.