View in EDS

CoCoA: A General Framework for Communication-Efficient Distributed Optimization.

Saved in:
Bibliographic Details
Title: CoCoA: A General Framework for Communication-Efficient Distributed Optimization.
Authors: Smith, Virginia1 SMITHV@STANFORD.EDU, Forte, Simone2 SIMONE.FORTE@GESS.ETHZ.CH, Chenxin Ma3 CHM514@LEHIGH.EDU, Takáč, Martin3 TAKAC.MT@GMAIL.COM, Jordan, Michael I.4 JORDAN@CS.BERKELEY.EDU, Jaggi, Martin5 MARTIN.JAGGI@EPFL.CH
Source: Journal of Machine Learning Research. 2018, Vol. 18 Issue 154-234, p1-49. 49p.
Subjects: Machine learning, Mathematical optimization, Convex functions, Stochastic convergence, Mathematical regularization
Abstract: The scale of modern datasets necessitates the development of efficient distributed optimization methods for machine learning. We present a general-purpose framework for distributed computing environments, CoCoA, that has an efficient communication scheme and is applicable to a wide variety of problems in machine learning and signal processing. We extend the framework to cover general non-strongly-convex regularizers, including L1-regularized problems like lasso, sparse logistic regression, and elastic net regularization, and show how earlier work can be derived as a special case. We provide convergence guarantees for the class of convex regularized loss minimization objectives, leveraging a novel approach in handling non-strongly-convex regularizers and non-smooth loss functions. The resulting framework has markedly improved performance over state-of-the-art methods, as we illustrate with an extensive set of experiments on real distributed datasets. [ABSTRACT FROM AUTHOR]
Copyright of Journal of Machine Learning Research is the property of Microtome Publishing and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Description
Abstract:The scale of modern datasets necessitates the development of efficient distributed optimization methods for machine learning. We present a general-purpose framework for distributed computing environments, CoCoA, that has an efficient communication scheme and is applicable to a wide variety of problems in machine learning and signal processing. We extend the framework to cover general non-strongly-convex regularizers, including L1-regularized problems like lasso, sparse logistic regression, and elastic net regularization, and show how earlier work can be derived as a special case. We provide convergence guarantees for the class of convex regularized loss minimization objectives, leveraging a novel approach in handling non-strongly-convex regularizers and non-smooth loss functions. The resulting framework has markedly improved performance over state-of-the-art methods, as we illustrate with an extensive set of experiments on real distributed datasets. [ABSTRACT FROM AUTHOR]
ISSN:15324435