Towards Formula Translation using Recursive Neural Networks
  While it has become common to perform automated translations on natural
language, performing translations between different representations of
mathematical formulae has thus far not been possible. We implemented the first
translator for mathematical formulae based on recursive neural networks. We
chose recursive neural networks because mathematical formulae inherently
include a structural encoding. In our implementation, we developed new
techniques and topologies for recursive tree-to-tree neural networks based on
multi-variate multi-valued Long Short-Term Memory cells. We propose a novel
approach for mini-batch training that utilizes clustering and tree traversal.
We evaluate our translator and analyze the behavior of our proposed topologies
and techniques based on a translation from generic LaTeX to the semantic LaTeX
notation. We use the semantic LaTeX notation from the Digital Library for
Mathematical Formulae and the Digital Repository for Mathematical Formulae at
the National Institute for Standards and Technology. We find that a simple
heuristics-based clustering algorithm outperforms the conventional clustering
algorithms on the task of clustering binary trees of mathematical formulae with
respect to their topology. Furthermore, we find a mask for the loss function,
which can prevent the neural network from finding a local minimum of the loss
function. Given our preliminary results, a complete translation from formula to
formula is not yet possible. However, we achieved a prediction accuracy of
47.05% for predicting symbols at the correct position and an accuracy of 92.3%
when ignoring the predicted position. Concluding, our work advances the field
of recursive neural networks by improving the training speed and quality of
training. In the future, we will work towards a complete translation allowing a
machine-interpretation of LaTeX formulae.
