Using PCA to Efficiently Represent State Spaces
  Reinforcement learning algorithms need to deal with the exponential growth of
states and actions when exploring optimal control in high-dimensional spaces.
This is known as the curse of dimensionality. By projecting the agent's state
onto a low-dimensional manifold, we can represent the state space in a smaller
and more efficient representation. By using this representation during
learning, the agent can converge to a good policy much faster. We test this
approach in the Mario Benchmarking Domain. When using dimensionality reduction
in Mario, learning converges much faster to a good policy. But, there is a
critical convergence-performance trade-off. By projecting onto a
low-dimensional manifold, we are ignoring important data. In this paper, we
explore this trade-off of convergence and performance. We find that learning in
as few as 4 dimensions (instead of 9), we can improve performance past learning
in the full dimensional space at a faster convergence rate.
