A Distributed Deep Reinforcement Learning Technique for Application
  Placement in Edge and Fog Computing Environments
  Fog/Edge computing is a novel computing paradigm supporting
resource-constrained Internet of Things (IoT) devices by the placement of their
tasks on the edge and/or cloud servers. Recently, several Deep Reinforcement
Learning (DRL)-based placement techniques have been proposed in fog/edge
computing environments, which are only suitable for centralized setups. The
training of well-performed DRL agents requires manifold training data while
obtaining training data is costly. Hence, these centralized DRL-based
techniques lack generalizability and quick adaptability, thus failing to
efficiently tackle application placement problems. Moreover, many IoT
applications are modeled as Directed Acyclic Graphs (DAGs) with diverse
topologies. Satisfying dependencies of DAG-based IoT applications incur
additional constraints and increase the complexity of placement problems. To
overcome these challenges, we propose an actor-critic-based distributed
application placement technique, working based on the IMPortance weighted
Actor-Learner Architectures (IMPALA). IMPALA is known for efficient distributed
experience trajectory generation that significantly reduces the exploration
costs of agents. Besides, it uses an adaptive off-policy correction method for
faster convergence to optimal solutions. Our technique uses recurrent layers to
capture temporal behaviors of input data and a replay buffer to improve the
sample efficiency. The performance results, obtained from simulation and
testbed experiments, demonstrate that our technique significantly improves the
execution cost of IoT applications up to 30\% compared to its counterparts.
