Evaluating and Characterizing Incremental Learning from Non-Stationary
  Data
  Incremental learning from non-stationary data poses special challenges to the
field of machine learning. Although new algorithms have been developed for
this, assessment of results and comparison of behaviors are still open
problems, mainly because evaluation metrics, adapted from more traditional
tasks, can be ineffective in this context. Overall, there is a lack of common
testing practices. This paper thus presents a testbed for incremental
non-stationary learning algorithms, based on specially designed synthetic
datasets. Also, test results are reported for some well-known algorithms to
show that the proposed methodology is effective at characterizing their
strengths and weaknesses. It is expected that this methodology will provide a
common basis for evaluating future contributions in the field.
