Predicting Algorithm Classes for Programming Word Problems
  We introduce the task of algorithm class prediction for programming word
problems. A programming word problem is a problem written in natural language,
which can be solved using an algorithm or a program. We define classes of
various programming word problems which correspond to the class of algorithms
required to solve the problem. We present four new datasets for this task, two
multiclass datasets with 550 and 1159 problems each and two multilabel datasets
having 3737 and 3960 problems each. We pose the problem as a text
classification problem and train neural network and non-neural network-based
models on this task. Our best performing classifier gets an accuracy of 62.7
percent for the multiclass case on the five class classification dataset,
Codeforces Multiclass-5 (CFMC5). We also do some human-level analysis and
compare human performance with that of our text classification models. Our best
classifier has an accuracy only 9 percent lower than that of a human on this
task. To the best of our knowledge, these are the first reported results on
such a task. We make our code and datasets publicly available.
