Optimal Passenger-Seeking Policies on E-hailing Platforms Using Markov
  Decision Process and Imitation Learning
  Vacant taxi drivers' passenger seeking process in a road network generates
additional vehicle miles traveled, adding congestion and pollution into the
road network and the environment. This paper aims to employ a Markov Decision
Process (MDP) to model idle e-hailing drivers' optimal sequential decisions in
passenger-seeking. Transportation network companies (TNC) or e-hailing (e.g.,
Didi, Uber) drivers exhibit different behaviors from traditional taxi drivers
because e-hailing drivers do not need to actually search for passengers.
Instead, they reposition themselves so that the matching platform can match a
passenger. Accordingly, we incorporate e-hailing drivers' new features into our
MDP model. The reward function used in the MDP model is uncovered by leveraging
an inverse reinforcement learning technique. We then use 44,160 Didi drivers'
3-day trajectories to train the model. To validate the effectiveness of the
model, a Monte Carlo simulation is conducted to simulate the performance of
drivers under the guidance of the optimal policy, which is then compared with
the performance of drivers following one baseline heuristic, namely, the local
hotspot strategy. The results show that our model is able to achieve a 17.5%
improvement over the local hotspot strategy in terms of the rate of return. The
proposed MDP model captures the supply-demand ratio considering the fact that
the number of drivers in this study is sufficiently large and thus the number
of unmatched orders is assumed to be negligible. To better incorporate the
competition among multiple drivers into the model, we have also devised and
calibrated a dynamic adjustment strategy of the order matching probability.
