Performance Dynamics and Termination Errors in Reinforcement Learning: A
  Unifying Perspective
  In reinforcement learning, a decision needs to be made at some point as to
whether it is worthwhile to carry on with the learning process or to terminate
it. In many such situations, stochastic elements are often present which govern
the occurrence of rewards, with the sequential occurrences of positive rewards
randomly interleaved with negative rewards. For most practical learners, the
learning is considered useful if the number of positive rewards always exceeds
the negative ones. A situation that often calls for learning termination is
when the number of negative rewards exceeds the number of positive rewards.
However, while this seems reasonable, the error of premature termination,
whereby termination is enacted along with the conclusion of learning failure
despite the positive rewards eventually far outnumber the negative ones, can be
significant. In this paper, using combinatorial analysis we study the error
probability in wrongly terminating a reinforcement learning activity which
undermines the effectiveness of an optimal policy, and we show that the
resultant error can be quite high. Whilst we demonstrate mathematically that
such errors can never be eliminated, we propose some practical mechanisms that
can effectively reduce such errors. Simulation experiments have been carried
out, the results of which are in close agreement with our theoretical findings.
