A Deep Reinforcement Learning Approach for Constrained Online Logistics
  Route Assignment
  As online shopping prevails and e-commerce platforms emerge, there is a
tremendous number of parcels being transported every day. Thus, it is crucial
for the logistics industry on how to assign a candidate logistics route for
each shipping parcel properly as it leaves a significant impact on the total
logistics cost optimization and business constraints satisfaction such as
transit hub capacity and delivery proportion of delivery providers. This online
route-assignment problem can be viewed as a constrained online decision-making
problem. Notably, the large amount (beyond ${10^5}$) of daily parcels, the
variability and non-Markovian characteristics of parcel information impose
difficulties on attaining (near-) optimal solution without violating
constraints excessively. In this paper, we develop a model-free DRL approach
named PPO-RA, in which Proximal Policy Optimization (PPO) is improved with
dedicated techniques to address the challenges for route assignment (RA). The
actor and critic networks use attention mechanism and parameter sharing to
accommodate each incoming parcel with varying numbers and identities of
candidate routes, without modeling non-Markovian parcel arriving dynamics since
we make assumption of i.i.d. parcel arrival. We use recorded delivery parcel
data to evaluate the performance of PPO-RA by comparing it with widely-used
baselines via simulation. The results show the capability of the proposed
approach to achieve considerable cost savings while satisfying most
constraints.
