Controlling Overestimation Bias with Truncated Mixture of Continuous
  Distributional Quantile Critics
  The overestimation bias is one of the major impediments to accurate
off-policy learning. This paper investigates a novel way to alleviate the
overestimation bias in a continuous control setting. Our method---Truncated
Quantile Critics, TQC,---blends three ideas: distributional representation of a
critic, truncation of critics prediction, and ensembling of multiple critics.
Distributional representation and truncation allow for arbitrary granular
overestimation control, while ensembling provides additional score
improvements. TQC outperforms the current state of the art on all environments
from the continuous control benchmark suite, demonstrating 25% improvement on
the most challenging Humanoid environment.
