Norm-Agnostic Linear Bandits
Linear bandits have a wide variety of applications including recommendation
systems yet they make one strong assumption: the algorithms must know an upper
bound $S$ on the norm of the unknown parameter $\theta^*$ that governs the
reward generation. Such an assumption forces the practitioner to guess $S$
involved in the confidence bound, leaving no choice but to wish that
$\|\theta^*\|\le S$ is true to guarantee that the regret will be low. In this
paper, we propose novel algorithms that do not require such knowledge for the
first time. Specifically, we propose two algorithms and analyze their regret
bounds: one for the changing arm set setting and the other for the fixed arm
set setting. Our regret bound for the former shows that the price of not
knowing $S$ does not affect the leading term in the regret bound and inflates
only the lower order term. For the latter, we do not pay any price in the
regret for now knowing $S$. Our numerical experiments show standard algorithms
assuming knowledge of $S$ can fail catastrophically when $\|\theta^*\|\le S$ is
not true whereas our algorithms enjoy low regret.