Optimal Actor-Critic Policy with Optimized Training Datasets
  Actor-critic (AC) algorithms are known for their efficacy and high
performance in solving reinforcement learning problems, but they also suffer
from low sampling efficiency. An AC based policy optimization process is
iterative and needs to frequently access the agent-environment system to
evaluate and update the policy by rolling out the policy, collecting rewards
and states (i.e. samples), and learning from them. It ultimately requires a
huge number of samples to learn an optimal policy. To improve sampling
efficiency, we propose a strategy to optimize the training dataset that
contains significantly less samples collected from the AC process. The dataset
optimization is made of a best episode only operation, a policy
parameter-fitness model, and a genetic algorithm module. The optimal policy
network trained by the optimized training dataset exhibits superior performance
compared to many contemporary AC algorithms in controlling autonomous dynamical
systems. Evaluation on standard benchmarks show that the method improves
sampling efficiency, ensures faster convergence to optima, and is more
data-efficient than its counterparts.
