Correlation-based construction of neighborhood and edge features
  Motivated by an abstract notion of low-level edge detector filters, we
propose a simple method of unsupervised feature construction based on pairwise
statistics of features. In the first step, we construct neighborhoods of
features by regrouping features that correlate. Then we use these subsets as
filters to produce new neighborhood features. Next, we connect neighborhood
features that correlate, and construct edge features by subtracting the
correlated neighborhood features of each other. To validate the usefulness of
the constructed features, we ran AdaBoost.MH on four multi-class classification
problems. Our most significant result is a test error of 0.94% on MNIST with an
algorithm which is essentially free of any image-specific priors. On CIFAR-10
our method is suboptimal compared to today's best deep learning techniques,
nevertheless, we show that the proposed method outperforms not only boosting on
the raw pixels, but also boosting on Haar filters.
