Teaching Machines to Code: Neural Markup Generation with Visual
  Attention
  We present a neural transducer model with visual attention that learns to
generate LaTeX markup of a real-world math formula given its image. Applying
sequence modeling and transduction techniques that have been very successful
across modalities such as natural language, image, handwriting, speech and
audio; we construct an image-to-markup model that learns to produce
syntactically and semantically correct LaTeX markup code over 150 words long
and achieves a BLEU score of 89%; improving upon the previous state-of-art for
the Im2Latex problem. We also demonstrate with heat-map visualization how
attention helps in interpreting the model and can pinpoint (detect and
localize) symbols on the image accurately despite having been trained without
any bounding box data.
