QuPeL: Quantized Personalization with Applications to Federated Learning
  Traditionally, federated learning (FL) aims to train a single global model
while collaboratively using multiple clients and a server. Two natural
challenges that FL algorithms face are heterogeneity in data across clients and
collaboration of clients with {\em diverse resources}. In this work, we
introduce a \textit{quantized} and \textit{personalized} FL algorithm QuPeL
that facilitates collective training with heterogeneous clients while
respecting resource diversity. For personalization, we allow clients to learn
\textit{compressed personalized models} with different quantization parameters
depending on their resources. Towards this, first we propose an algorithm for
learning quantized models through a relaxed optimization problem, where
quantization values are also optimized over. When each client participating in
the (federated) learning process has different requirements of the quantized
model (both in value and precision), we formulate a quantized personalization
framework by introducing a penalty term for local client objectives against a
globally trained model to encourage collaboration. We develop an alternating
proximal gradient update for solving this quantized personalization problem,
and we analyze its convergence properties. Numerically, we show that optimizing
over the quantization levels increases the performance and we validate that
QuPeL outperforms both FedAvg and local training of clients in a heterogeneous
setting.
