Entropy based Independent Learning in Anonymous Multi-Agent Settings
  Efficient sequential matching of supply and demand is a problem of interest
in many online to offline services. For instance, Uber, Lyft, Grab for matching
taxis to customers; Ubereats, Deliveroo, FoodPanda etc for matching restaurants
to customers. In these online to offline service problems, individuals who are
responsible for supply (e.g., taxi drivers, delivery bikes or delivery van
drivers) earn more by being at the "right" place at the "right" time. We are
interested in developing approaches that learn to guide individuals to be in
the "right" place at the "right" time (to maximize revenue) in the presence of
other similar "learning" individuals and only local aggregated observation of
other agents states (e.g., only number of other taxis in same zone as current
agent).
  A key characteristic of the domains of interest is that the interactions
between individuals are anonymous, i.e., the outcome of an interaction
(competing for demand) is dependent only on the number and not on the identity
of the agents. We model these problems using the Anonymous MARL (AyMARL) model.
The key contribution of this paper is in employing principle of maximum entropy
to provide a general framework of independent learning that is both empirically
effective (even with only local aggregated information of agent population
distribution) and theoretically justified.
  Finally, our approaches provide a significant improvement with respect to
joint and individual revenue on a generic simulator for online to offline
services and a real world taxi problem over existing approaches. More
importantly, this is achieved while having the least variance in revenues
earned by the learning individuals, an indicator of fairness.
