Asynchronous Decentralized Learning of a Neural Network
  In this work, we exploit an asynchronous computing framework namely ARock to
learn a deep neural network called self-size estimating feedforward neural
network (SSFN) in a decentralized scenario. Using this algorithm namely
asynchronous decentralized SSFN (dSSFN), we provide the centralized equivalent
solution under certain technical assumptions. Asynchronous dSSFN relaxes the
communication bottleneck by allowing one node activation and one side
communication, which reduces the communication overhead significantly,
consequently increasing the learning speed. We compare asynchronous dSSFN with
traditional synchronous dSSFN in the experimental results, which shows the
competitive performance of asynchronous dSSFN, especially when the
communication network is sparse.
