Placeto: Learning Generalizable Device Placement Algorithms for
  Distributed Machine Learning
  We present Placeto, a reinforcement learning (RL) approach to efficiently
find device placements for distributed neural network training. Unlike prior
approaches that only find a device placement for a specific computation graph,
Placeto can learn generalizable device placement policies that can be applied
to any graph. We propose two key ideas in our approach: (1) we represent the
policy as performing iterative placement improvements, rather than outputting a
placement in one shot; (2) we use graph embeddings to capture relevant
information about the structure of the computation graph, without relying on
node labels for indexing. These ideas allow Placeto to train efficiently and
generalize to unseen graphs. Our experiments show that Placeto requires up to
6.1x fewer training steps to find placements that are on par with or better
than the best placements found by prior approaches. Moreover, Placeto is able
to learn a generalizable placement policy for any given family of graphs, which
can then be used without any retraining to predict optimized placements for
unseen graphs from the same family. This eliminates the large overhead incurred
by prior RL approaches whose lack of generalizability necessitates re-training
from scratch every time a new graph is to be placed.
