Impression Space from Deep Template Network
  It is an innate ability for humans to imagine something only according to
their impression, without having to memorize all the details of what they have
seen. In this work, we would like to demonstrate that a trained convolutional
neural network also has the capability to "remember" its input images. To
achieve this, we propose a simple but powerful framework to establish an
{\emph{Impression Space}} upon an off-the-shelf pretrained network. This
network is referred to as the {\emph{Template Network}} because its filters
will be used as templates to reconstruct images from the impression. In our
framework, the impression space and image space are bridged by a layer-wise
encoding and iterative decoding process. It turns out that the impression space
indeed captures the salient features from images, and it can be directly
applied to tasks such as unpaired image translation and image synthesis through
impression matching without further network training. Furthermore, the
impression naturally constructs a high-level common space for different data.
Based on this, we propose a mechanism to model the data relations inside the
impression space, which is able to reveal the feature similarity between
images. Our code will be released.
