Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: DS-MLR: Exploiting Double Separability for Scaling up Distributed Multinomial Logistic Regression
(Submitted on 16 Apr 2016 (v1), revised 31 Mar 2017 (this version, v2), latest version 3 Aug 2018 (v7))
Abstract: Multinomial logistic regression is a popular tool in the arsenal of machine learning algorithms, yet scaling it to datasets with very large number of data points and classes has not been trivial. This is primarily because one needs to compute the log-partition function on every data point. This makes distributing the computation hard. In this paper, we present a distributed stochastic gradient descent based optimization method (DS-MLR) for scaling up multinomial logistic regression problems to very large data. Our algorithm exploits double-separability, an attractive property we observe in the objective functions of several models in machine learning, that allows us to achieve both data as well as model parallelism simultaneously. In addition to being parallelizable, our algorithm can also easily be made asynchronous. In order to demonstrate the effectiveness of our method, we solve a very large multi-class classification problem on the reddit dataset with data and parameter sizes of 200 GB and 300 GB respectively. Such a scale of data calls for simultaneous data and model parallelism which is where DS-MLR fits in.
Submission history
From: Parameswaran Raman [view email][v1] Sat, 16 Apr 2016 07:26:58 GMT (3347kb,D)
[v2] Fri, 31 Mar 2017 18:45:59 GMT (3320kb,D)
[v3] Tue, 23 May 2017 08:06:02 GMT (2899kb,D)
[v4] Thu, 15 Feb 2018 01:02:54 GMT (2585kb,D)
[v5] Wed, 18 Apr 2018 01:15:04 GMT (2586kb,D)
[v6] Mon, 21 May 2018 23:44:36 GMT (2701kb,D)
[v7] Fri, 3 Aug 2018 22:13:06 GMT (2701kb,D)
Link back to: arXiv, form interface, contact.