Using Subjective Logic to Estimate Uncertainty in Multi-Armed Bandit
  Problems
  The multi-armed bandit problem is a classical decision-making problem where
an agent has to learn an optimal action balancing exploration and exploitation.
Properly managing this trade-off requires a correct assessment of uncertainty;
in multi-armed bandits, as in other machine learning applications, it is
important to distinguish between stochasticity that is inherent to the system
(aleatoric uncertainty) and stochasticity that derives from the limited
knowledge of the agent (epistemic uncertainty). In this paper we consider the
formalism of subjective logic, a concise and expressive framework to express
Dirichlet-multinomial models as subjective opinions, and we apply it to the
problem of multi-armed bandits. We propose new algorithms grounded in
subjective logic to tackle the multi-armed bandit problem, we compare them
against classical algorithms from the literature, and we analyze the insights
they provide in evaluating the dynamics of uncertainty. Our preliminary results
suggest that subjective logic quantities enable useful assessment of
uncertainty that may be exploited by more refined agents.
