Reusability and Transferability of Macro Actions for Reinforcement
  Learning
  Conventional reinforcement learning (RL) typically determines an appropriate
primitive action at each timestep. However, by using a proper macro action,
defined as a sequence of primitive actions, an agent is able to bypass
intermediate states to a farther state and facilitate its learning procedure.
The problem we would like to investigate is what associated beneficial
properties that macro actions may possess. In this paper, we unveil the
properties of reusability and transferability of macro actions. The first
property, reusability, means that a macro action generated along with one RL
method can be reused by another RL method for training, while the second one,
transferability, means that a macro action can be utilized for training agents
in similar environments with different reward settings. In our experiments, we
first generate macro actions along with RL methods. We then provide a set of
analyses to reveal the properties of reusability and transferability of the
generated macro actions.
