Bottom-Up Meta-Policy Search
  Despite of the recent progress in agents that learn through interaction,
there are several challenges in terms of sample efficiency and generalization
across unseen behaviors during training. To mitigate these problems, we propose
and apply a first-order Meta-Learning algorithm called Bottom-Up Meta-Policy
Search (BUMPS), which works with two-phase optimization procedure: firstly, in
a meta-training phase, it distills few expert policies to create a meta-policy
capable of generalizing knowledge to unseen tasks during training; secondly, it
applies a fast adaptation strategy named Policy Filtering, which evaluates few
policies sampled from the meta-policy distribution and selects which best
solves the task. We conducted all experiments in the RoboCup 3D Soccer
Simulation domain, in the context of kick motion learning. We show that, given
our experimental setup, BUMPS works in scenarios where simple multi-task
Reinforcement Learning does not. Finally, we performed experiments in a way to
evaluate each component of the algorithm.
