An Efficient Algorithm For Generalized Linear Bandit: Online Stochastic
  Gradient Descent and Thompson Sampling
  We consider the contextual bandit problem, where a player sequentially makes
decisions based on past observations to maximize the cumulative reward.
Although many algorithms have been proposed for contextual bandit, most of them
rely on finding the maximum likelihood estimator at each iteration, which
requires $O(t)$ time at the $t$-th iteration and are memory inefficient. A
natural way to resolve this problem is to apply online stochastic gradient
descent (SGD) so that the per-step time and memory complexity can be reduced to
constant with respect to $t$, but a contextual bandit policy based on online
SGD updates that balances exploration and exploitation has remained elusive. In
this work, we show that online SGD can be applied to the generalized linear
bandit problem. The proposed SGD-TS algorithm, which uses a single-step SGD
update to exploit past information and uses Thompson Sampling for exploration,
achieves $\tilde{O}(\sqrt{T})$ regret with the total time complexity that
scales linearly in $T$ and $d$, where $T$ is the total number of rounds and $d$
is the number of features. Experimental results show that SGD-TS consistently
outperforms existing algorithms on both synthetic and real datasets.
