Recklessly Approximate Sparse Coding
  It has recently been observed that certain extremely simple feature encoding
techniques are able to achieve state of the art performance on several standard
image classification benchmarks including deep belief networks, convolutional
nets, factored RBMs, mcRBMs, convolutional RBMs, sparse autoencoders and
several others. Moreover, these "triangle" or "soft threshold" encodings are
ex- tremely efficient to compute. Several intuitive arguments have been put
forward to explain this remarkable performance, yet no mathematical
justification has been offered.
  The main result of this report is to show that these features are realized as
an approximate solution to the a non-negative sparse coding problem. Using this
connection we describe several variants of the soft threshold features and
demonstrate their effectiveness on two image classification benchmark tasks.
