Feeling of Presence Maximization: mmWave-Enabled Virtual Reality Meets
  Deep Reinforcement Learning
  This paper investigates the problem of providing ultra-reliable and
energy-efficient virtual reality (VR) experiences for wireless mobile users. To
ensure reliable ultra-high-definition (UHD) video frame delivery to mobile
users and enhance their immersive visual experiences, a coordinated multipoint
(CoMP) transmission technique and millimeter wave (mmWave) communications are
exploited. Owing to user movement and time-varying wireless channels, the
wireless VR experience enhancement problem is formulated as a
sequence-dependent and mixed-integer problem with a goal of maximizing users'
feeling of presence (FoP) in the virtual world, subject to power consumption
constraints on access points (APs) and users' head-mounted displays (HMDs). The
problem, however, is hard to be directly solved due to the lack of users'
accurate tracking information and the sequence-dependent and mixed-integer
characteristics. To overcome this challenge, we develop a parallel echo state
network (ESN) learning method to predict users' tracking information by
training fresh and historical tracking samples separately collected by APs.
With the learnt results, we propose a deep reinforcement learning (DRL) based
optimization algorithm to solve the formulated problem. In this algorithm, we
implement deep neural networks (DNNs) as a scalable solution to produce integer
decision variables and solving a continuous power control problem to criticize
the integer decision variables. Finally, the performance of the proposed
algorithm is compared with various benchmark algorithms, and the impact of
different design parameters is also discussed. Simulation results demonstrate
that the proposed algorithm is more 4.14% energy-efficient than the benchmark
algorithms.
