Heterogeneous Federated Learning
  Federated learning learns from scattered data by fusing collaborative models
from local nodes. However, due to chaotic information distribution, the model
fusion may suffer from structural misalignment with regard to unmatched
parameters. In this work, we propose a novel federated learning framework to
resolve this issue by establishing a firm structure-information alignment
across collaborative models. Specifically, we design a feature-oriented
regulation method ({$\Psi$-Net}) to ensure explicit feature information
allocation in different neural network structures. Applying this regulating
method to collaborative models, matchable structures with similar feature
information can be initialized at the very early training stage. During the
federated learning process under either IID or non-IID scenarios, dedicated
collaboration schemes further guarantee ordered information distribution with
definite structure matching, so as the comprehensive model alignment.
Eventually, this framework effectively enhances the federated learning
applicability to extensive heterogeneous settings, while providing excellent
convergence speed, accuracy, and computation/communication efficiency.
