Cross-Silo Federated Learning for Multi-Tier Networks with Vertical and
  Horizontal Data Partitioning
  We consider federated learning in tiered communication networks. Our network
model consists of a set of silos, each holding a vertical partition of the
data. Each silo contains a hub and a set of clients, with the silo's vertical
data shard partitioned horizontally across its clients. We propose Tiered
Decentralized Coordinate Descent (TDCD), a communication-efficient
decentralized training algorithm for such two-tiered networks. The clients in
each silo perform multiple local gradient steps before sharing updates with
their hub to reduce communication overhead. Each hub adjusts its coordinates by
averaging its workers' updates, and then hubs exchange intermediate updates
with one another. We present a theoretical analysis of our algorithm and show
the dependence of the convergence rate on the number of vertical partitions and
the number of local updates. We further validate our approach empirically via
simulation-based experiments using a variety of datasets and objectives.
