Combinatorial Optimization by Graph Pointer Networks and Hierarchical
  Reinforcement Learning
  In this work, we introduce Graph Pointer Networks (GPNs) trained using
reinforcement learning (RL) for tackling the traveling salesman problem (TSP).
GPNs build upon Pointer Networks by introducing a graph embedding layer on the
input, which captures relationships between nodes. Furthermore, to approximate
solutions to constrained combinatorial optimization problems such as the TSP
with time windows, we train hierarchical GPNs (HGPNs) using RL, which learns a
hierarchical policy to find an optimal city permutation under constraints. Each
layer of the hierarchy is designed with a separate reward function, resulting
in stable training. Our results demonstrate that GPNs trained on small-scale
TSP50/100 problems generalize well to larger-scale TSP500/1000 problems, with
shorter tour lengths and faster computational times. We verify that for
constrained TSP problems such as the TSP with time windows, the feasible
solutions found via hierarchical RL training outperform previous baselines. In
the spirit of reproducible research we make our data, models, and code publicly
available.
