Equilibrium Inverse Reinforcement Learning for Ride-hailing Vehicle
  Network
  Ubiquitous mobile computing have enabled ride-hailing services to collect
vast amounts of behavioral data of riders and drivers and optimize supply and
demand matching in real time. While these mobility service providers have some
degree of control over the market by assigning vehicles to requests, they need
to deal with the uncertainty arising from self-interested driver behavior since
workers are usually free to drive when they are not assigned tasks. In this
work, we formulate the problem of passenger-vehicle matching in a sparsely
connected graph and proposed an algorithm to derive an equilibrium policy in a
multi-agent environment. Our framework combines value iteration methods to
estimate the optimal policy given expected state visitation and policy
propagation to compute multi-agent state visitation frequencies. Furthermore,
we developed a method to learn the driver's reward function transferable to an
environment with significantly different dynamics from training data. We
evaluated the robustness to changes in spatio-temporal supply-demand
distributions and deterioration in data quality using a real-world taxi
trajectory dataset; our approach significantly outperforms several baselines in
terms of imitation accuracy. The computational time required to obtain an
equilibrium policy shared by all vehicles does not depend on the number of
agents, and even on the scale of real-world services, it takes only a few
seconds on a single CPU.
