Processing Network Controls via Deep Reinforcement Learning
  Novel advanced policy gradient (APG) algorithms, such as proximal policy
optimization (PPO), trust region policy optimization, and their variations,
have become the dominant reinforcement learning (RL) algorithms because of
their ease of implementation and good practical performance. This dissertation
is concerned with theoretical justification and practical application of the
APG algorithms for solving processing network control optimization problems.
Processing network control problems are typically formulated as Markov decision
process (MDP) or semi-Markov decision process (SMDP) problems that have several
unconventional for RL features: infinite state spaces, unbounded costs,
long-run average cost objectives. Policy improvement bounds play a crucial role
in the theoretical justification of the APG algorithms. In this thesis we
refine existing bounds for MDPs with finite state spaces and prove novel policy
improvement bounds for classes of MDPs and SMDPs used to model processing
network operations. We consider two examples of processing network control
problems and customize the PPO algorithm to solve them. First, we consider
parallel-server and multiclass queueing networks controls. Second, we consider
the drivers repositioning problem in a ride-hailing service system. For both
examples the PPO algorithm with auxiliary modifications consistently generates
control policies that outperform state-of-art heuristics.
