The Perils of Exploration under Competition: A Computational Modeling
  Approach
  We empirically study the interplay between exploration and competition.
Systems that learn from interactions with users often engage in exploration:
making potentially suboptimal decisions in order to acquire new information for
future decisions. However, when multiple systems are competing for the same
market of users, exploration may hurt a system's reputation in the near term,
with adverse competitive effects. In particular, a system may enter a "death
spiral", when the short-term reputation cost decreases the number of users for
the system to learn from, which degrades its performance relative to
competition and further decreases its market share.
  We ask whether better exploration algorithms are incentivized under
competition. We run extensive numerical experiments in a stylized duopoly model
in which two firms deploy multi-armed bandit algorithms and compete for myopic
users. We find that duopoly and monopoly tend to favor a primitive "greedy
algorithm" that does not explore and leads to low consumer welfare, whereas a
temporary monopoly (a duopoly with an early entrant) may incentivize better
bandit algorithms and lead to higher consumer welfare. Our findings shed light
on the first-mover advantage in the digital economy by exploring the role that
data can play as a barrier to entry in online markets.
