Multi-echelon Supply Chains with Uncertain Seasonal Demands and Lead
  Times Using Deep Reinforcement Learning
  We address the problem of production planning and distribution in
multi-echelon supply chains. We consider uncertain demands and lead times which
makes the problem stochastic and non-linear. A Markov Decision Process
formulation and a Non-linear Programming model are presented. As a sequential
decision-making problem, Deep Reinforcement Learning (RL) is a possible
solution approach. This type of technique has gained a lot of attention from
Artificial Intelligence and Optimization communities in recent years.
Considering the good results obtained with Deep RL approaches in different
areas there is a growing interest in applying them in problems from the
Operations Research field. We have used a Deep RL technique, namely Proximal
Policy Optimization (PPO2), to solve the problem considering uncertain, regular
and seasonal demands and constant or stochastic lead times. Experiments are
carried out in different scenarios to better assess the suitability of the
algorithm. An agent based on a linearized model is used as a baseline.
Experimental results indicate that PPO2 is a competitive and adequate tool for
this type of problem. PPO2 agent is better than baseline in all scenarios with
stochastic lead times (7.3-11.2%), regardless of whether demands are seasonal
or not. In scenarios with constant lead times, the PPO2 agent is better when
uncertain demands are non-seasonal (2.2-4.7%). The results show that the
greater the uncertainty of the scenario, the greater the viability of this type
of approach.
