We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Multinomial Logistic Regression Algorithms via Quadratic Gradient

Authors: John Chiang
Abstract: Multinomial logistic regression, also known by other names such as multiclass logistic regression and softmax regression, is a fundamental classification method that generalizes binary logistic regression to multiclass problems. A recently work proposed a faster gradient called $\texttt{quadratic gradient}$ that can accelerate the binary logistic regression training, and presented an enhanced Nesterov's accelerated gradient (NAG) method for binary logistic regression.
In this paper, we extend this work to multiclass logistic regression and propose an enhanced Adaptive Gradient Algorithm (Adagrad) that can accelerate the original Adagrad method. We test the enhanced NAG method and the enhanced Adagrad method on some multiclass-problem datasets. Experimental results show that both enhanced methods converge faster than their original ones respectively.
Comments: There is a good chance that the enhanced gradient methods for multiclass LR could be used in the classisation neural-network training via the softmax activation and the cross-entropy loss
Subjects: Machine Learning (cs.LG); Optimization and Control (math.OC)
Cite as: arXiv:2208.06828 [cs.LG]
  (or arXiv:2208.06828v1 [cs.LG] for this version)

Submission history

From: John Chiang [view email]
[v1] Sun, 14 Aug 2022 11:00:27 GMT (1083kb,D)

Link back to: arXiv, form interface, contact.