Holarchic Structures for Decentralized Deep Learning - A Performance
  Analysis
  Structure plays a key role in learning performance. In centralized
computational systems, hyperparameter optimization and regularization
techniques such as dropout are computational means to enhance learning
performance by adjusting the deep hierarchical structure. However, in
decentralized deep learning by the Internet of Things, the structure is an
actual network of autonomous interconnected devices such as smart phones that
interact via complex network protocols. Self-adaptation of the learning
structure is a challenge. Uncertainties such as network latency, node and link
failures or even bottlenecks by limited processing capacity and energy
availability can signif- icantly downgrade learning performance. Network
self-organization and self-management is complex, while it requires additional
computational and network resources that hinder the feasibility of
decentralized deep learning. In contrast, this paper introduces a self-adaptive
learning approach based on holarchic learning structures for exploring,
mitigating and boosting learning performance in distributed environments with
uncertainties. A large-scale performance analysis with 864000 experiments fed
with synthetic and real-world data from smart grid and smart city pilot
projects confirm the cost-effectiveness of holarchic structures for
decentralized deep learning.
