Lifelong Learning with a Changing Action Set
  In many real-world sequential decision making problems, the number of
available actions (decisions) can vary over time. While problems like
catastrophic forgetting, changing transition dynamics, changing rewards
functions, etc. have been well-studied in the lifelong learning literature, the
setting where the action set changes remains unaddressed. In this paper, we
present an algorithm that autonomously adapts to an action set whose size
changes over time. To tackle this open problem, we break it into two problems
that can be solved iteratively: inferring the underlying, unknown, structure in
the space of actions and optimizing a policy that leverages this structure. We
demonstrate the efficiency of this approach on large-scale real-world lifelong
learning problems.
