Federated User Representation Learning
  Collaborative personalization, such as through learned user representations
(embeddings), can improve the prediction accuracy of neural-network-based
models significantly. We propose Federated User Representation Learning (FURL),
a simple, scalable, privacy-preserving and resource-efficient way to utilize
existing neural personalization techniques in the Federated Learning (FL)
setting. FURL divides model parameters into federated and private parameters.
Private parameters, such as private user embeddings, are trained locally, but
unlike federated parameters, they are not transferred to or averaged on the
server. We show theoretically that this parameter split does not affect
training for most model personalization approaches. Storing user embeddings
locally not only preserves user privacy, but also improves memory locality of
personalization compared to on-server training. We evaluate FURL on two
datasets, demonstrating a significant improvement in model quality with 8% and
51% performance increases, and approximately the same level of performance as
centralized training with only 0% and 4% reductions. Furthermore, we show that
user embeddings learned in FL and the centralized setting have a very similar
structure, indicating that FURL can learn collaboratively through the shared
parameters while preserving user privacy.
