Online 3D Bin Packing with Constrained Deep Reinforcement Learning
  We solve a challenging yet practically useful variant of 3D Bin Packing
Problem (3D-BPP). In our problem, the agent has limited information about the
items to be packed into the bin, and an item must be packed immediately after
its arrival without buffering or readjusting. The item's placement also
subjects to the constraints of collision avoidance and physical stability. We
formulate this online 3D-BPP as a constrained Markov decision process. To solve
the problem, we propose an effective and easy-to-implement constrained deep
reinforcement learning (DRL) method under the actor-critic framework. In
particular, we introduce a feasibility predictor to predict the feasibility
mask for the placement actions and use it to modulate the action probabilities
output by the actor during training. Such supervisions and transformations to
DRL facilitate the agent to learn feasible policies efficiently. Our method can
also be generalized e.g., with the ability to handle lookahead or items with
different orientations. We have conducted extensive evaluation showing that the
learned policy significantly outperforms the state-of-the-art methods. A user
study suggests that our method attains a human-level performance.
