HeterPS: Distributed Deep Learning With Reinforcement Learning Based
  Scheduling in Heterogeneous Environments
  Deep neural networks (DNNs) exploit many layers and a large number of
parameters to achieve excellent performance. The training process of DNN models
generally handles large-scale input data with many sparse features, which
incurs high Input/Output (IO) cost, while some layers are compute-intensive.
The training process generally exploits distributed computing resources to
reduce training time. In addition, heterogeneous computing resources, e.g.,
CPUs, GPUs of multiple types, are available for the distributed training
process. Thus, the scheduling of multiple layers to diverse computing resources
is critical for the training process. To efficiently train a DNN model using
the heterogeneous computing resources, we propose a distributed framework,
i.e., Paddle-Heterogeneous Parameter Server (Paddle-HeterPS), composed of a
distributed architecture and a Reinforcement Learning (RL)-based scheduling
method. The advantages of Paddle-HeterPS are three-fold compared with existing
frameworks. First, Paddle-HeterPS enables efficient training process of diverse
workloads with heterogeneous computing resources. Second, Paddle-HeterPS
exploits an RL-based method to efficiently schedule the workload of each layer
to appropriate computing resources to minimize the cost while satisfying
throughput constraints. Third, Paddle-HeterPS manages data storage and data
communication among distributed computing resources. We carry out extensive
experiments to show that Paddle-HeterPS significantly outperforms
state-of-the-art approaches in terms of throughput (14.5 times higher) and
monetary cost (312.3% smaller). The codes of the framework are publicly
available at: https://github.com/PaddlePaddle/Paddle.
