We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Electrical Engineering and Systems Science > Signal Processing

Title: Bounds on mutual information of mixture data for classification tasks

Abstract: The data for many classification problems, such as pattern and speech recognition, follow mixture distributions. To quantify the optimum performance for classification tasks, the Shannon mutual information is a natural information-theoretic metric, as it is directly related to the probability of error. The mutual information between mixture data and the class label does not have an analytical expression, nor any efficient computational algorithms. We introduce a variational upper bound, a lower bound, and three estimators, all employing pair-wise divergences between mixture components. We compare the new bounds and estimators with Monte Carlo stochastic sampling and bounds derived from entropy bounds. To conclude, we evaluate the performance of the bounds and estimators through numerical simulations.
Subjects: Signal Processing (eess.SP)
Cite as: arXiv:2101.11670 [eess.SP]
  (or arXiv:2101.11670v1 [eess.SP] for this version)

Submission history

From: Yijun Ding [view email]
[v1] Wed, 27 Jan 2021 20:11:26 GMT (2856kb,D)

Link back to: arXiv, form interface, contact.