The Engagement-Diversity Connection: Evidence from a Field Experiment on
  Spotify
  It remains unknown whether personalized recommendations increase or decrease
the diversity of content people consume. We present results from a randomized
field experiment on Spotify testing the effect of personalized recommendations
on consumption diversity. In the experiment, both control and treatment users
were given podcast recommendations, with the sole aim of increasing podcast
consumption. Treatment users' recommendations were personalized based on their
music listening history, whereas control users were recommended popular
podcasts among users in their demographic group. We find that, on average, the
treatment increased podcast streams by 28.90%. However, the treatment also
decreased the average individual-level diversity of podcast streams by 11.51%,
and increased the aggregate diversity of podcast streams by 5.96%, indicating
that personalized recommendations have the potential to create patterns of
consumption that are homogenous within and diverse across users, a pattern
reflecting Balkanization. Our results provide evidence of an
"engagement-diversity trade-off" when recommendations are optimized solely to
drive consumption: while personalized recommendations increase user engagement,
they also affect the diversity of consumed content. This shift in consumption
diversity can affect user retention and lifetime value, and impact the optimal
strategy for content producers. We also observe evidence that our treatment
affected streams from sections of Spotify's app not directly affected by the
experiment, suggesting that exposure to personalized recommendations can affect
the content that users consume organically. We believe these findings highlight
the need for academics and practitioners to continue investing in
personalization methods that explicitly take into account the diversity of
content recommended.
