Hierarchical Deep Reinforcement Learning Approach for Multi-Objective
  Scheduling With Varying Queue Sizes
  Multi-objective task scheduling (MOTS) is the task scheduling while
optimizing multiple and possibly contradicting constraints. A challenging
extension of this problem occurs when every individual task is a
multi-objective optimization problem by itself. While deep reinforcement
learning (DRL) has been successfully applied to complex sequential problems,
its application to the MOTS domain has been stymied by two challenges. The
first challenge is the inability of the DRL algorithm to ensure that every item
is processed identically regardless of its position in the queue. The second
challenge is the need to manage large queues, which results in large neural
architectures and long training times. In this study we present MERLIN, a
robust, modular and near-optimal DRL-based approach for multi-objective task
scheduling. MERLIN applies a hierarchical approach to the MOTS problem by
creating one neural network for the processing of individual tasks and another
for the scheduling of the overall queue. In addition to being smaller and with
shorted training times, the resulting architecture ensures that an item is
processed in the same manner regardless of its position in the queue.
Additionally, we present a novel approach for efficiently applying DRL-based
solutions on very large queues, and demonstrate how we effectively scale MERLIN
to process queue sizes that are larger by orders of magnitude than those on
which it was trained. Extensive evaluation on multiple queue sizes show that
MERLIN outperforms multiple well-known baselines by a large margin (>22%).
