References & Citations
Computer Science > Networking and Internet Architecture
Title: Flexible Parallel Learning in Edge Scenarios: Communication, Computational and Energy Cost
(Submitted on 19 Jan 2022)
Abstract: Traditionally, distributed machine learning takes the guise of (i) different nodes training the same model (as in federated learning), or (ii) one model being split among multiple nodes (as in distributed stochastic gradient descent). In this work, we highlight how fog- and IoT-based scenarios often require combining both approaches, and we present a framework for flexible parallel learning (FPL), achieving both data and model parallelism. Further, we investigate how different ways of distributing and parallelizing learning tasks across the participating nodes result in different computation, communication, and energy costs. Our experiments, carried out using state-of-the-art deep-network architectures and large-scale datasets, confirm that FPL allows for an excellent trade-off among computational (hence energy) cost, communication overhead, and learning performance.
Submission history
From: Francesco Malandrino [view email][v1] Wed, 19 Jan 2022 03:47:04 GMT (2331kb,D)
Link back to: arXiv, form interface, contact.