Learning Representation from Neural Fisher Kernel with Low-rank
  Approximation
  In this paper, we study the representation of neural networks from the view
of kernels. We first define the Neural Fisher Kernel (NFK), which is the Fisher
Kernel applied to neural networks. We show that NFK can be computed for both
supervised and unsupervised learning models, which can serve as a unified tool
for representation extraction. Furthermore, we show that practical NFKs exhibit
low-rank structures. We then propose an efficient algorithm that computes a low
rank approximation of NFK, which scales to large datasets and networks. We show
that the low-rank approximation of NFKs derived from unsupervised generative
models and supervised learning models gives rise to high-quality compact
representations of data, achieving competitive results on a variety of machine
learning tasks.
