We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Constructing Multiclass Classifiers using Binary Classifiers Under Log-Loss

Abstract: The construction of multiclass classifiers from binary elements is studied in this paper, and performance is quantified by the regret, defined with respect to the Bayes optimal log-loss. We discuss two known methods. The first is one vs. all (OVA), for which we prove that the multiclass regret is upper bounded by the sum of binary regrets of the constituent classifiers. The second is hierarchical classification, based on a binary tree. For this method we prove that the multiclass regret is exactly a weighted sum of constituent binary regrets where the weighing is determined by the tree structure.
We also introduce a leverage-hierarchical classification method, which potentially yields smaller log-loss and regret. The advantages of these classification methods are demonstrated by simulation on both synthetic and real-life datasets.
Comments: A shorter version of this contribution was presented in ISIT 2021
Subjects: Machine Learning (cs.LG); Information Theory (cs.IT); Statistics Theory (math.ST); Machine Learning (stat.ML)
Cite as: arXiv:2102.08184 [cs.LG]
  (or arXiv:2102.08184v2 [cs.LG] for this version)

Submission history

From: Assaf Ben-Yishai [view email]
[v1] Tue, 16 Feb 2021 14:34:59 GMT (139kb,D)
[v2] Thu, 12 Aug 2021 12:29:05 GMT (187kb,D)

Link back to: arXiv, form interface, contact.