Collaborative Learning over Wireless Networks: An Introductory Overview
  In this chapter, we will mainly focus on collaborative training across
wireless devices. Training a ML model is equivalent to solving an optimization
problem, and many distributed optimization algorithms have been developed over
the last decades. These distributed ML algorithms provide data locality; that
is, a joint model can be trained collaboratively while the data available at
each participating device remains local. This addresses, to some extend, the
privacy concern. They also provide computational scalability as they allow
exploiting computational resources distributed across many edge devices.
However, in practice, this does not directly lead to a linear gain in the
overall learning speed with the number of devices. This is partly due to the
communication bottleneck limiting the overall computation speed. Additionally,
wireless devices are highly heterogeneous in their computational capabilities,
and both their computation speed and communication rate can be highly
time-varying due to physical factors. Therefore, distributed learning
algorithms, particularly those to be implemented at the wireless network edge,
must be carefully designed taking into account the impact of time-varying
communication network as well as the heterogeneous and stochastic computation
capabilities of devices.
