An Empirical Analysis on Transparent Algorithmic Exploration in
  Recommender Systems
  All learning algorithms for recommendations face inevitable and critical
trade-off between exploiting partial knowledge of a user's preferences for
short-term satisfaction and exploring additional user preferences for long-term
coverage. Although exploration is indispensable for long success of a
recommender system, the exploration has been considered as the risk to decrease
user satisfaction. The reason for the risk is that items chosen for exploration
frequently mismatch with the user's interests. To mitigate this risk,
recommender systems have mixed items chosen for exploration into a
recommendation list, disguising the items as recommendations to elicit feedback
on the items to discover the user's additional tastes. This mix-in approach has
been widely used in many recommenders, but there is rare research, evaluating
the effectiveness of the mix-in approach or proposing a new approach for
eliciting user feedback without deceiving users. In this work, we aim to
propose a new approach for feedback elicitation without any deception and
compare our approach to the conventional mix-in approach for evaluation. To
this end, we designed a recommender interface that reveals which items are for
exploration and conducted a within-subject study with 94 MTurk workers. Our
results indicated that users left significantly more feedback on items chosen
for exploration with our interface. Besides, users evaluated that our new
interface is better than the conventional mix-in interface in terms of novelty,
diversity, transparency, trust, and satisfaction. Finally, path analysis show
that, in only our new interface, exploration caused to increase user-centric
evaluation metrics. Our work paves the way for how to design an interface,
which utilizes learning algorithm based on users' feedback signals, giving
better user experience and gathering more feedback data.
