The Peaking Phenomenon in Semi-supervised Learning
  For the supervised least squares classifier, when the number of training
objects is smaller than the dimensionality of the data, adding more data to the
training set may first increase the error rate before decreasing it. This,
possibly counterintuitive, phenomenon is known as peaking. In this work, we
observe that a similar but more pronounced version of this phenomenon also
occurs in the semi-supervised setting, where instead of labeled objects,
unlabeled objects are added to the training set. We explain why the learning
curve has a more steep incline and a more gradual decline in this setting
through simulation studies and by applying an approximation of the learning
curve based on the work by Raudys & Duin.
