Linear Stochastic Bandits Under Safety Constraints
  Bandit algorithms have various application in safety-critical systems, where
it is important to respect the system constraints that rely on the bandit's
unknown parameters at every round. In this paper, we formulate a linear
stochastic multi-armed bandit problem with safety constraints that depend
(linearly) on an unknown parameter vector. As such, the learner is unable to
identify all safe actions and must act conservatively in ensuring that her
actions satisfy the safety constraint at all rounds (at least with high
probability). For these bandits, we propose a new UCB-based algorithm called
Safe-LUCB, which includes necessary modifications to respect safety
constraints. The algorithm has two phases. During the pure exploration phase
the learner chooses her actions at random from a restricted set of safe actions
with the goal of learning a good approximation of the entire unknown safe set.
Once this goal is achieved, the algorithm begins a safe
exploration-exploitation phase where the learner gradually expands their
estimate of the set of safe actions while controlling the growth of regret. We
provide a general regret bound for the algorithm, as well as a problem
dependent bound that is connected to the location of the optimal action within
the safe set. We then propose a modified heuristic that exploits our problem
dependent analysis to improve the regret.
