Deep Reinforcement Learning for Stochastic Computation Offloading in
  Digital Twin Networks
  The rapid development of Industrial Internet of Things (IIoT) requires
industrial production towards digitalization to improve network efficiency.
Digital Twin is a promising technology to empower the digital transformation of
IIoT by creating virtual models of physical objects. However, the provision of
network efficiency in IIoT is very challenging due to resource-constrained
devices, stochastic tasks, and resources heterogeneity. Distributed resources
in IIoT networks can be efficiently exploited through computation offloading to
reduce energy consumption while enhancing data processing efficiency. In this
paper, we first propose a new paradigm Digital Twin Networks (DTN) to build
network topology and the stochastic task arrival model in IIoT systems. Then,
we formulate the stochastic computation offloading and resource allocation
problem to minimize the long-term energy efficiency. As the formulated problem
is a stochastic programming problem, we leverage Lyapunov optimization
technique to transform the original problem into a deterministic per-time slot
problem. Finally, we present Asynchronous Actor-Critic (AAC) algorithm to find
the optimal stochastic computation offloading policy. Illustrative results
demonstrate that our proposed scheme is able to significantly outperforms the
benchmarks.
