Distributed Reinforcement Learning for Privacy-Preserving Dynamic Edge
  Caching
  Mobile edge computing (MEC) is a prominent computing paradigm which expands
the application fields of wireless communication. Due to the limitation of the
capacities of user equipments and MEC servers, edge caching (EC) optimization
is crucial to the effective utilization of the caching resources in MEC-enabled
wireless networks. However, the dynamics and complexities of content
popularities over space and time as well as the privacy preservation of users
pose significant challenges to EC optimization. In this paper, a
privacy-preserving distributed deep deterministic policy gradient (P2D3PG)
algorithm is proposed to maximize the cache hit rates of devices in the MEC
networks. Specifically, we consider the fact that content popularities are
dynamic, complicated and unobservable, and formulate the maximization of cache
hit rates on devices as distributed problems under the constraints of privacy
preservation. In particular, we convert the distributed optimizations into
distributed model-free Markov decision process problems and then introduce a
privacy-preserving federated learning method for popularity prediction.
Subsequently, a P2D3PG algorithm is developed based on distributed
reinforcement learning to solve the distributed problems. Simulation results
demonstrate the superiority of the proposed approach in improving EC hit rate
over the baseline methods while preserving user privacy.
