Large-Margin kNN Classification Using a Deep Encoder Network
  KNN is one of the most popular classification methods, but it often fails to
work well with inappropriate choice of distance metric or due to the presence
of numerous class-irrelevant features. Linear feature transformation methods
have been widely applied to extract class-relevant information to improve kNN
classification, which is very limited in many applications. Kernels have been
used to learn powerful non-linear feature transformations, but these methods
fail to scale to large datasets. In this paper, we present a scalable
non-linear feature mapping method based on a deep neural network pretrained
with restricted boltzmann machines for improving kNN classification in a
large-margin framework, which we call DNet-kNN. DNet-kNN can be used for both
classification and for supervised dimensionality reduction. The experimental
results on two benchmark handwritten digit datasets show that DNet-kNN has much
better performance than large-margin kNN using a linear mapping and kNN based
on a deep autoencoder pretrained with retricted boltzmann machines.
