Competing Bandits: The Perils of Exploration Under Competition
  Most online platforms strive to learn from interactions with users, and many
engage in exploration: making potentially suboptimal choices for the sake of
acquiring new information. We study the interplay between exploration and
competition: how such platforms balance the exploration for learning and the
competition for users. Here users play three distinct roles: they are customers
that generate revenue, they are sources of data for learning, and they are
self-interested agents which choose among the competing platforms.
  We consider a stylized duopoly model in which two firms face the same
multi-armed bandit problem. Users arrive one by one and choose between the two
firms, so that each firm makes progress on its bandit problem only if it is
chosen. Through a mix of theoretical results and numerical simulations, we
study whether and to what extent competition incentivizes the adoption of
better bandit algorithms, and whether it leads to welfare increases for users.
We find that stark competition induces firms to commit to a "greedy" bandit
algorithm that leads to low welfare. However, weakening competition by
providing firms with some "free" users incentivizes better exploration
strategies and increases welfare. We investigate two channels for weakening the
competition: relaxing the rationality of users and giving one firm a
first-mover advantage. Our findings are closely related to the "competition vs.
innovation" relationship, and elucidate the first-mover advantage in the
digital economy.
