Solving the Order Batching and Sequencing Problem using Deep
  Reinforcement Learning
  In e-commerce markets, on time delivery is of great importance to customer
satisfaction. In this paper, we present a Deep Reinforcement Learning (DRL)
approach for deciding how and when orders should be batched and picked in a
warehouse to minimize the number of tardy orders. In particular, the technique
facilitates making decisions on whether an order should be picked individually
(pick-by-order) or picked in a batch with other orders (pick-by-batch), and if
so with which other orders. We approach the problem by formulating it as a
semi-Markov decision process and develop a vector-based state representation
that includes the characteristics of the warehouse system. This allows us to
create a deep reinforcement learning solution that learns a strategy by
interacting with the environment and solve the problem with a proximal policy
optimization algorithm. We evaluate the performance of the proposed DRL
approach by comparing it with several batching and sequencing heuristics in
different problem settings. The results show that the DRL approach is able to
develop a strategy that produces consistent, good solutions and performs better
than the proposed heuristics.
