Combining Model-Free Q-Ensembles and Model-Based Approaches for Informed
  Exploration
  Q-Ensembles are a model-free approach where input images are fed into
different Q-networks and exploration is driven by the assumption that
uncertainty is proportional to the variance of the output Q-values obtained.
They have been shown to perform relatively well compared to other exploration
strategies. Further, model-based approaches, such as encoder-decoder models
have been used successfully for next frame prediction given previous frames.
This paper proposes to integrate the model-free Q-ensembles and model-based
approaches with the hope of compounding the benefits of both and achieving
superior exploration as a result. Results show that a model-based trajectory
memory approach when combined with Q-ensembles produces superior performance
when compared to only using Q-ensembles.
