An Empirical Comparison of Syllabuses for Curriculum Learning
  Syllabuses for curriculum learning have been developed on an ad-hoc, per task
basis and little is known about the relative performance of different
syllabuses. We identify a number of syllabuses used in the literature. We
compare the identified syllabuses based on their effect on the speed of
learning and generalization ability of a LSTM network on three sequential
learning tasks. We find that the choice of syllabus has limited effect on the
generalization ability of a trained network. In terms of speed of learning our
results demonstrate that the best syllabus is task dependent but that a
recently proposed automated curriculum learning approach - Predictive Gain,
performs very competitively against all identified hand-crafted syllabuses. The
best performing hand-crafted syllabus which we term Look Back and Forward
combines a syllabus which steps through tasks in the order of their difficulty
with a uniform distribution over all tasks. Our experimental results provide an
empirical basis for the choice of syllabus on a new problem that could benefit
from curriculum learning. Additionally, insights derived from our results shed
light on how to successfully design new syllabuses.
