Pournaras, E, Yadhunathan, S and Diaconescu, A (2020) Holarchic structures for decentralized deep learning: a performance analysis. Cluster Computing, 23 (1). pp. 219-240. ISSN 1386-7857
Abstract
Structure plays a key role in learning performance. In centralized computational systems, hyperparameter optimization and regularization techniques such as dropout are computational means to enhance learning performance by adjusting the deep hierarchical structure. However, in decentralized deep learning by the Internet of Things, the structure is an actual network of autonomous interconnected devices such as smart phones that interact via complex network protocols. Self-adaptation of the learning structure is a challenge. Uncertainties such as network latency, node and link failures or even bottlenecks by limited processing capacity and energy availability can significantly downgrade learning performance. Network self-organization and self-management is complex, while it requires additional computational and network resources that hinder the feasibility of decentralized deep learning. In contrast, this paper introduces a self-adaptive learning approach based on holarchic learning structures for exploring, mitigating and boosting learning performance in distributed environments with uncertainties. A large-scale performance analysis with 864,000 experiments fed with synthetic and real-world data from smart grid and smart city pilot projects confirm the cost-effectiveness of holarchic structures for decentralized deep learning.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | (c) The Author(s) 2019. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. |
Keywords: | Deep learning; Optimization; Holarchy; Multi-agent system; Resilience; Smart city |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) > School of Computing (Leeds) |
Depositing User: | Symplectic Publications |
Date Deposited: | 18 Feb 2020 12:32 |
Last Modified: | 11 Mar 2020 08:47 |
Status: | Published |
Publisher: | Springer |
Identification Number: | 10.1007/s10586-019-02906-4 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:157102 |
Download
Filename: Pournaras2020_Article_HolarchicStructuresForDecentra.pdf
Licence: CC-BY 4.0