Deep reinforcement learning with a particle dynamics environment applied
  to emergency evacuation of a room with obstacles
  A very successful model for simulating emergency evacuation is the
social-force model. At the heart of the model is the self-driven force that is
applied to an agent and is directed towards the exit. However, it is not clear
if the application of this force results in optimal evacuation, especially in
complex environments with obstacles. Here, we develop a deep reinforcement
learning algorithm in association with the social force model to train agents
to find the fastest evacuation path. During training, we penalize every step of
an agent in the room and give zero reward at the exit. We adopt the Dyna-Q
learning approach. We first show that in the case of a room without obstacles
the resulting self-driven force points directly towards the exit as in the
social force model and that the median exit time intervals calculated using the
two methods are not significantly different. Then, we investigate evacuation of
a room with one obstacle and one exit. We show that our method produces similar
results with the social force model when the obstacle is convex. However, in
the case of concave obstacles, which sometimes can act as traps for agents
governed purely by the social force model and prohibit complete room
evacuation, our approach is clearly advantageous since it derives a policy that
results in object avoidance and complete room evacuation without additional
assumptions. We also study evacuation of a room with multiple exits. We show
that agents are able to evacuate efficiently from the nearest exit through a
shared network trained for a single agent. Finally, we test the robustness of
the Dyna-Q learning approach in a complex environment with multiple exits and
obstacles. Overall, we show that our model can efficiently simulate emergency
evacuation in complex environments with multiple room exits and obstacles where
it is difficult to obtain an intuitive rule for fast evacuation.
