We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.ST

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Statistics Theory

Title: Minimax Estimation of the $L_1$ Distance

Abstract: We consider the problem of estimating the $L_1$ distance between two discrete probability measures $P$ and $Q$ from empirical data in a nonasymptotic and large alphabet setting. When $Q$ is known and one obtains $n$ samples from $P$, we show that for every $Q$, the minimax rate-optimal estimator with $n$ samples achieves performance comparable to that of the maximum likelihood estimator (MLE) with $n\ln n$ samples. When both $P$ and $Q$ are unknown, we construct minimax rate-optimal estimators whose worst case performance is essentially that of the known $Q$ case with $Q$ being uniform, implying that $Q$ being uniform is essentially the most difficult case. The \emph{effective sample size enlargement} phenomenon, identified in Jiao \emph{et al.} (2015), holds both in the known $Q$ case for every $Q$ and the $Q$ unknown case. However, the construction of optimal estimators for $\|P-Q\|_1$ requires new techniques and insights beyond the approximation-based method of functional estimation in Jiao \emph{et al.} (2015).
Comments: to appear on IEEE Transactions on Information Theory
Subjects: Statistics Theory (math.ST); Information Theory (cs.IT)
DOI: 10.1109/TIT.2018.2846245
Cite as: arXiv:1705.00807 [math.ST]
  (or arXiv:1705.00807v7 [math.ST] for this version)

Submission history

From: Jiantao Jiao [view email]
[v1] Tue, 2 May 2017 06:03:06 GMT (51kb)
[v2] Wed, 24 May 2017 02:23:57 GMT (51kb)
[v3] Mon, 17 Jul 2017 08:32:15 GMT (53kb)
[v4] Tue, 18 Jul 2017 02:15:06 GMT (53kb)
[v5] Sun, 10 Sep 2017 18:23:00 GMT (53kb)
[v6] Mon, 14 May 2018 11:51:28 GMT (208kb,D)
[v7] Sat, 23 Jun 2018 17:33:10 GMT (209kb,D)

Link back to: arXiv, form interface, contact.