Secure Distributed/Federated Learning: Prediction-Privacy Trade-Off for
  Multi-Agent System
  Decentralized learning is an efficient emerging paradigm for boosting the
computing capability of multiple bounded computing agents. In the big data era,
performing inference within the distributed and federated learning (DL and FL)
frameworks, the central server needs to process a large amount of data while
relying on various agents to perform multiple distributed training tasks.
Considering the decentralized computing topology, privacy has become a
first-class concern. Moreover, assuming limited information processing
capability for the agents calls for a sophisticated \textit{privacy-preserving
decentralization} that ensures efficient computation. Towards this end, we
study the \textit{privacy-aware server to multi-agent assignment} problem
subject to information processing constraints associated with each agent, while
maintaining the privacy and assuring learning informative messages received by
agents about a global terminal through the distributed private federated
learning (DPFL) approach. To find a decentralized scheme for a two-agent
system, we formulate an optimization problem that balances privacy and
accuracy, taking into account the quality of compression constraints associated
with each agent. We propose an iterative converging algorithm by alternating
over self-consistent equations. We also numerically evaluate the proposed
solution to show the privacy-prediction trade-off and demonstrate the efficacy
of the novel approach in ensuring privacy in DL and FL.
