Benchmarking Online Sequence-to-Sequence and Character-based Handwriting
  Recognition from IMU-Enhanced Pens
  Handwriting is one of the most frequently occurring patterns in everyday life
and with it come challenging applications such as handwriting recognition
(HWR), writer identification, and signature verification. In contrast to
offline HWR that only uses spatial information (i.e., images), online HWR
(OnHWR) uses richer spatio-temporal information (i.e., trajectory data or
inertial data). While there exist many offline HWR datasets, there is only
little data available for the development of OnHWR methods as it requires
hardware-integrated pens. This paper presents data and benchmark models for
real-time sequence-to-sequence (seq2seq) learning and single character-based
recognition. Our data is recorded by a sensor-enhanced ballpoint pen, yielding
sensor data streams from triaxial accelerometers, a gyroscope, a magnetometer
and a force sensor at 100Hz. We propose a variety of datasets including
equations and words for both the writer-dependent and writer-independent tasks.
We provide an evaluation benchmark for seq2seq and single character-based HWR
using recurrent and temporal convolutional networks and Transformers combined
with a connectionist temporal classification (CTC) loss and cross entropy
losses. Our methods do not resort to language or lexicon models.
