Value Function is All You Need: A Unified Learning Framework for Ride
  Hailing Platforms
  Large ride-hailing platforms, such as DiDi, Uber and Lyft, connect tens of
thousands of vehicles in a city to millions of ride demands throughout the day,
providing great promises for improving transportation efficiency through the
tasks of order dispatching and vehicle repositioning. Existing studies,
however, usually consider the two tasks in simplified settings that hardly
address the complex interactions between the two, the real-time fluctuations
between supply and demand, and the necessary coordinations due to the
large-scale nature of the problem. In this paper we propose a unified
value-based dynamic learning framework (V1D3) for tackling both tasks. At the
center of the framework is a globally shared value function that is updated
continuously using online experiences generated from real-time platform
transactions. To improve the sample-efficiency and the robustness, we further
propose a novel periodic ensemble method combining the fast online learning
with a large-scale offline training scheme that leverages the abundant
historical driver trajectory data. This allows the proposed framework to adapt
quickly to the highly dynamic environment, to generalize robustly to recurrent
patterns and to drive implicit coordinations among the population of managed
vehicles. Extensive experiments based on real-world datasets show considerably
improvements over other recently proposed methods on both tasks. Particularly,
V1D3 outperforms the first prize winners of both dispatching and repositioning
tracks in the KDD Cup 2020 RL competition, achieving state-of-the-art results
on improving both total driver income and user experience related metrics.
