Lie Access Neural Turing Machine
  Following the recent trend in explicit neural memory structures, we present a
new design of an external memory, wherein memories are stored in an Euclidean
key space $\mathbb R^n$. An LSTM controller performs read and write via
specialized read and write heads. It can move a head by either providing a new
address in the key space (aka random access) or moving from its previous
position via a Lie group action (aka Lie access). In this way, the "L" and "R"
instructions of a traditional Turing Machine are generalized to arbitrary
elements of a fixed Lie group action. For this reason, we name this new model
the Lie Access Neural Turing Machine, or LANTM.
  We tested two different configurations of LANTM against an LSTM baseline in
several basic experiments. We found the right configuration of LANTM to
outperform the baseline in all of our experiments. In particular, we trained
LANTM on addition of $k$-digit numbers for $2 \le k \le 16$, but it was able to
generalize almost perfectly to $17 \le k \le 32$, all with the number of
parameters 2 orders of magnitude below the LSTM baseline.
