Personalized Federated Learning with First Order Model Optimization
  While federated learning traditionally aims to train a single global model
across decentralized local datasets, one model may not always be ideal for all
participating clients. Here we propose an alternative, where each client only
federates with other relevant clients to obtain a stronger model per
client-specific objectives. To achieve this personalization, rather than
computing a single model average with constant weights for the entire
federation as in traditional FL, we efficiently calculate optimal weighted
model combinations for each client, based on figuring out how much a client can
benefit from another's model. We do not assume knowledge of any underlying data
distributions or client similarities, and allow each client to optimize for
arbitrary target distributions of interest, enabling greater flexibility for
personalization. We evaluate and characterize our method on a variety of
federated settings, datasets, and degrees of local data heterogeneity. Our
method outperforms existing alternatives, while also enabling new features for
personalized FL such as transfer outside of local data distributions.
