Jointly-Learned State-Action Embedding for Efficient Reinforcement
  Learning
  While reinforcement learning has achieved considerable successes in recent
years, state-of-the-art models are often still limited by the size of state and
action spaces. Model-free reinforcement learning approaches use some form of
state representations and the latest work has explored embedding techniques for
actions, both with the aim of achieving better generalization and
applicability. However, these approaches consider only states or actions,
ignoring the interaction between them when generating embedded representations.
In this work, we establish the theoretical foundations for the validity of
training a reinforcement learning agent using embedded states and actions. We
then propose a new approach for jointly learning embeddings for states and
actions that combines aspects of model-free and model-based reinforcement
learning, which can be applied in both discrete and continuous domains.
Specifically, we use a model of the environment to obtain embeddings for states
and actions and present a generic architecture that leverages these to learn a
policy. In this way, the embedded representations obtained via our approach
enable better generalization over both states and actions by capturing
similarities in the embedding spaces. Evaluations of our approach on several
gaming, robotic control, and recommender systems show it significantly
outperforms state-of-the-art models in both discrete/continuous domains with
large state/action spaces, thus confirming its efficacy.
