Hindsight Experience Replay with Kronecker Product Approximate Curvature
  Hindsight Experience Replay (HER) is one of the efficient algorithm to solve
Reinforcement Learning tasks related to sparse rewarded environments.But due to
its reduced sample efficiency and slower convergence HER fails to perform
effectively. Natural gradients solves these challenges by converging the model
parameters better. It avoids taking bad actions that collapse the training
performance. However updating parameters in neural networks requires expensive
computation and thus increase in training time. Our proposed method solves the
above mentioned challenges with better sample efficiency and faster convergence
with increased success rate. A common failure mode for DDPG is that the learned
Q-function begins to dramatically overestimate Q-values, which then leads to
the policy breaking, because it exploits the errors in the Q-function. We solve
this issue by including Twin Delayed Deep Deterministic Policy Gradients(TD3)
in HER. TD3 learns two Q-functions instead of one and it adds noise tothe
target action, to make it harder for the policy to exploit Q-function errors.
The experiments are done with the help of OpenAis Mujoco environments. Results
on these environments show that our algorithm (TDHER+KFAC) performs better
inmost of the scenarios
