Dealing with Sparse Rewards Using Graph Neural Networks
  Deep reinforcement learning in partially observable environments is a
difficult task in itself, and can be further complicated by a sparse reward
signal. Most tasks involving navigation in three-dimensional environments
provide the agent with extremely limited information. Typically, the agent
receives a visual observation input from the environment and is rewarded once
at the end of the episode. A good reward function could substantially improve
the convergence of reinforcement learning algorithms for such tasks. The
classic approach to increase the density of the reward signal is to augment it
with supplementary rewards. This technique is called the reward shaping. In
this study, we propose two modifications of one of the recent reward shaping
methods based on graph convolutional networks: the first involving advanced
aggregation functions, and the second utilizing the attention mechanism. We
empirically validate the effectiveness of our solutions for the task of
navigation in a 3D environment with sparse rewards. For the solution featuring
attention mechanism, we are also able to show that the learned attention is
concentrated on edges corresponding to important transitions in 3D environment.
