Fully automatic scoring of handwritten descriptive answers in Japanese
  language tests
  This paper presents an experiment of automatically scoring handwritten
descriptive answers in the trial tests for the new Japanese university entrance
examination, which were made for about 120,000 examinees in 2017 and 2018.
There are about 400,000 answers with more than 20 million characters. Although
all answers have been scored by human examiners, handwritten characters are not
labelled. We present our attempt to adapt deep neural network-based handwriting
recognizers trained on a labelled handwriting dataset into this unlabeled
answer set. Our proposed method combines different training strategies,
ensembles multiple recognizers, and uses a language model built from a large
general corpus to avoid overfitting into specific data. In our experiment, the
proposed method records character accuracy of over 97% using about 2,000
verified labelled answers that account for less than 0.5% of the dataset. Then,
the recognized answers are fed into a pre-trained automatic scoring system
based on the BERT model without correcting misrecognized characters and
providing rubric annotations. The automatic scoring system achieves from 0.84
to 0.98 of Quadratic Weighted Kappa (QWK). As QWK is over 0.8, it represents
acceptable similarity of scoring between the automatic scoring system and the
human examiners. These results are promising for further research on end-to-end
automatic scoring of descriptive answers.
