A Hybrid Architecture for Federated and Centralized Learning
  Many of the machine learning tasks rely on centralized learning (CL), which
requires the transmission of local datasets from the clients to a parameter
server (PS) entailing huge communication overhead. To overcome this, federated
learning (FL) has been suggested as a promising tool, wherein the clients send
only the model updates to the PS instead of the whole dataset. However, FL
demands powerful computational resources from the clients. In practice, not all
the clients have sufficient computational resources to participate in training.
To address this common scenario, we propose a more efficient approach called
hybrid federated and centralized learning (HFCL), wherein only the clients with
sufficient resources employ FL, while the remaining ones send their datasets to
the PS, which computes the model on behalf of them. Then, the model parameters
are aggregated at the PS. To improve the efficiency of dataset transmission, we
propose two different techniques: i) increased computation-per-client and ii)
sequential data transmission. Notably, the HFCL frameworks outperform FL with
up to 20\% improvement in the learning accuracy when only half of the clients
perform FL while having 50\% less communication overhead than CL since all the
clients collaborate on the learning process with their datasets.
