FedMM: Saddle Point Optimization for Federated Adversarial Domain
  Adaptation
  Federated adversary domain adaptation is a unique distributed minimax
training task due to the prevalence of label imbalance among clients, with each
client only seeing a subset of the classes of labels required to train a global
model. To tackle this problem, we propose a distributed minimax optimizer
referred to as FedMM, designed specifically for the federated adversary domain
adaptation problem. It works well even in the extreme case where each client
has different label classes and some clients only have unsupervised tasks. We
prove that FedMM ensures convergence to a stationary point with domain-shifted
unsupervised data. On a variety of benchmark datasets, extensive experiments
show that FedMM consistently achieves either significant communication savings
or significant accuracy improvements over federated optimizers based on the
gradient descent ascent (GDA) algorithm. When training from scratch, for
example, it outperforms other GDA based federated average methods by around
$20\%$ in accuracy over the same communication rounds; and it consistently
outperforms when training from pre-trained models with an accuracy improvement
from $5.4\%$ to $9\%$ for different networks.
