Learning from a Learning User for Optimal Recommendations
  In real-world recommendation problems, especially those with a formidably
large item space, users have to gradually learn to estimate the utility of any
fresh recommendations from their experience about previously consumed items.
This in turn affects their interaction dynamics with the system and can
invalidate previous algorithms built on the omniscient user assumption. In this
paper, we formalize a model to capture such "learning users" and design an
efficient system-side learning solution, coined Noise-Robust Active Ellipsoid
Search (RAES), to confront the challenges brought by the non-stationary
feedback from such a learning user. Interestingly, we prove that the regret of
RAES deteriorates gracefully as the convergence rate of user learning becomes
worse, until reaching linear regret when the user's learning fails to converge.
Experiments on synthetic datasets demonstrate the strength of RAES for such a
contemporaneous system-user learning problem. Our study provides a novel
perspective on modeling the feedback loop in recommendation problems.
