Resource Constrained Deep Reinforcement Learning
  In urban environments, supply resources have to be constantly matched to the
"right" locations (where customer demand is present) so as to improve quality
of life. For instance, ambulances have to be matched to base stations regularly
so as to reduce response time for emergency incidents in EMS (Emergency
Management Systems); vehicles (cars, bikes, scooters etc.) have to be matched
to docking stations so as to reduce lost demand in shared mobility systems.
Such problem domains are challenging owing to the demand uncertainty,
combinatorial action spaces (due to allocation) and constraints on allocation
of resources (e.g., total resources, minimum and maximum number of resources at
locations and regions).
  Existing systems typically employ myopic and greedy optimization approaches
to optimize allocation of supply resources to locations. Such approaches
typically are unable to handle surges or variances in demand patterns well.
Recent research has demonstrated the ability of Deep RL methods in adapting
well to highly uncertain environments. However, existing Deep RL methods are
unable to handle combinatorial action spaces and constraints on allocation of
resources. To that end, we have developed three approaches on top of the well
known actor critic approach, DDPG (Deep Deterministic Policy Gradient) that are
able to handle constraints on resource allocation. More importantly, we
demonstrate that they are able to outperform leading approaches on simulators
validated on semi-real and real data sets.
