Prediction and Generalisation over Directed Actions by Grid Cells
  Knowing how the effects of directed actions generalise to new situations
(e.g. moving North, South, East and West, or turning left, right, etc.) is key
to rapid generalisation across new situations. Markovian tasks can be
characterised by a state space and a transition matrix and recent work has
proposed that neural grid codes provide an efficient representation of the
state space, as eigenvectors of a transition matrix reflecting diffusion across
states, that allows efficient prediction of future state distributions. Here we
extend the eigenbasis prediction model, utilising tools from Fourier analysis,
to prediction over arbitrary translation-invariant directed transition
structures (i.e. displacement and diffusion), showing that a single set of
eigenvectors can support predictions over arbitrary directed actions via
action-specific eigenvalues. We show how to define a "sense of direction" to
combine actions to reach a target state (ignoring task-specific deviations from
translation-invariance), and demonstrate that adding the Fourier
representations to a deep Q network aids policy learning in continuous control
tasks. We show the equivalence between the generalised prediction framework and
traditional models of grid cell firing driven by self-motion to perform path
integration, either using oscillatory interference (via Fourier components as
velocity-controlled oscillators) or continuous attractor networks (via analysis
of the update dynamics). We thus provide a unifying framework for the role of
the grid system in predictive planning, sense of direction and path
integration: supporting generalisable inference over directed actions across
different tasks.
