Hyperparameter Optimization and Boosting for Classifying Facial
  Expressions: How good can a "Null" Model be?
  One of the goals of the ICML workshop on representation and learning is to
establish benchmark scores for a new data set of labeled facial expressions.
This paper presents the performance of a "Null" model consisting of
convolutions with random weights, PCA, pooling, normalization, and a linear
readout. Our approach focused on hyperparameter optimization rather than novel
model components. On the Facial Expression Recognition Challenge held by the
Kaggle website, our hyperparameter optimization approach achieved a score of
60% accuracy on the test data. This paper also introduces a new ensemble
construction variant that combines hyperparameter optimization with the
construction of ensembles. This algorithm constructed an ensemble of four
models that scored 65.5% accuracy. These scores rank 12th and 5th respectively
among the 56 challenge participants. It is worth noting that our approach was
developed prior to the release of the data set, and applied without
modification; our strong competition performance suggests that the TPE
hyperparameter optimization algorithm and domain expertise encoded in our Null
model can generalize to new image classification data sets.
