Model Selection for Generic Contextual Bandits
  We consider the problem of model selection for the general stochastic
contextual bandits under the realizability assumption. We propose a successive
refinement based algorithm called Adaptive Contextual Bandit ({\ttfamily ACB}),
that works in phases and successively eliminates model classes that are too
simple to fit the given instance. We prove that this algorithm is adaptive,
i.e., the regret rate order-wise matches that of {\ttfamily FALCON}, the
state-of-art contextual bandit algorithm of Levi et. al '20, that needs
knowledge of the true model class. The price of not knowing the correct model
class is only an additive term contributing to the second order term in the
regret bound. This cost possess the intuitive property that it becomes smaller
as the model class becomes easier to identify, and vice-versa. We then show
that a much simpler explore-then-commit (ETC) style algorithm also obtains a
regret rate of matching that of {\ttfamily FALCON}, despite not knowing the
true model class. However, the cost of model selection is higher in ETC as
opposed to in {\ttfamily ACB}, as expected. Furthermore, {\ttfamily ACB}
applied to the linear bandit setting with unknown sparsity, order-wise recovers
the model selection guarantees previously established by algorithms tailored to
the linear setting.
