TornadoAggregate: Accurate and Scalable Federated Learning via the
  Ring-Based Architecture
  Federated learning has emerged as a new paradigm of collaborative machine
learning; however, many prior studies have used global aggregation along a star
topology without much consideration of the communication scalability or the
diurnal property relied on clients' local time variety. In contrast, ring
architecture can resolve the scalability issue and even satisfy the diurnal
property by iterating nodes without an aggregation. Nevertheless, such
ring-based algorithms can inherently suffer from the high-variance problem. To
this end, we propose a novel algorithm called TornadoAggregate that improves
both accuracy and scalability by facilitating the ring architecture. In
particular, to improve the accuracy, we reformulate the loss minimization into
a variance reduction problem and establish three principles to reduce variance:
Ring-Aware Grouping, Small Ring, and Ring Chaining. Experimental results show
that TornadoAggregate improved the test accuracy by up to 26.7% and achieved
near-linear scalability.
