Contextual Multi-armed Bandit Algorithm for Semiparametric Reward Model
  Contextual multi-armed bandit (MAB) algorithms have been shown promising for
maximizing cumulative rewards in sequential decision tasks such as news article
recommendation systems, web page ad placement algorithms, and mobile health.
However, most of the proposed contextual MAB algorithms assume linear
relationships between the reward and the context of the action. This paper
proposes a new contextual MAB algorithm for a relaxed, semiparametric reward
model that supports nonstationarity. The proposed method is less restrictive,
easier to implement and faster than two alternative algorithms that consider
the same model, while achieving a tight regret upper bound. We prove that the
high-probability upper bound of the regret incurred by the proposed algorithm
has the same order as the Thompson sampling algorithm for linear reward models.
The proposed and existing algorithms are evaluated via simulation and also
applied to Yahoo! news article recommendation log data.
