Q-Learning with Differential Entropy of Q-Tables
  It is well-known that information loss can occur in the classic and simple
Q-learning algorithm. Entropy-based policy search methods were introduced to
replace Q-learning and to design algorithms that are more robust against
information loss. We conjecture that the reduction in performance during
prolonged training sessions of Q-learning is caused by a loss of information,
which is non-transparent when only examining the cumulative reward without
changing the Q-learning algorithm itself. We introduce Differential Entropy of
Q-tables (DE-QT) as an external information loss detector to the Q-learning
algorithm. The behaviour of DE-QT over training episodes is analyzed to find an
appropriate stopping criterion during training. The results reveal that DE-QT
can detect the most appropriate stopping point, where a balance between a high
success rate and a high efficiency is met for classic Q-Learning algorithm.
