A Teacher-Student Markov Decision Process-based Framework for Online
  Correctional Learning
  A classical learning setting typically concerns an agent/student who collects
data, or observations, from a system in order to estimate a certain property of
interest. Correctional learning is a type of cooperative teacher-student
framework where a teacher, who has partial knowledge about the system, has the
ability to observe and alter (correct) the observations received by the student
in order to improve the accuracy of its estimate. In this paper, we show how
the variance of the estimate of the student can be reduced with the help of the
teacher. We formulate the corresponding online problem - where the teacher has
to decide, at each time instant, whether or not to change the observations due
to a limited budget - as a Markov decision process, from which the optimal
policy is derived using dynamic programming. We validate the framework in
numerical experiments, and compare the optimal online policy with the one from
the batch setting.
