Decentralized SGD with Over-the-Air Computation
  We study the performance of decentralized stochastic gradient descent (DSGD)
in a wireless network, where the nodes collaboratively optimize an objective
function using their local datasets. Unlike the conventional setting, where the
nodes communicate over error-free orthogonal communication links, we assume
that transmissions are prone to additive noise and interference.We first
consider a point-to-point (P2P) transmission strategy, termed the OAC-P2P
scheme, in which the node pairs are scheduled in an orthogonal fashion to
minimize interference. Since in the DSGD framework, each node requires a linear
combination of the neighboring models at the consensus step, we then propose
the OAC-MAC scheme, which utilizes the signal superposition property of the
wireless medium to achieve over-the-air computation (OAC). For both schemes, we
cast the scheduling problem as a graph coloring problem. We numerically
evaluate the performance of these two schemes for the MNIST image
classification task under various network conditions. We show that the OAC-MAC
scheme attains better convergence performance with a fewer communication
rounds.
