Ultrafast Photorealistic Style Transfer via Neural Architecture Search
  The key challenge in photorealistic style transfer is that an algorithm
should faithfully transfer the style of a reference photo to a content photo
while the generated image should look like one captured by a camera. Although
several photorealistic style transfer algorithms have been proposed, they need
to rely on post- and/or pre-processing to make the generated images look
photorealistic. If we disable the additional processing, these algorithms would
fail to produce plausible photorealistic stylization in terms of detail
preservation and photorealism. In this work, we propose an effective solution
to these issues. Our method consists of a construction step (C-step) to build a
photorealistic stylization network and a pruning step (P-step) for
acceleration. In the C-step, we propose a dense auto-encoder named PhotoNet
based on a carefully designed pre-analysis. PhotoNet integrates a feature
aggregation module (BFA) and instance normalized skip links (INSL). To generate
faithful stylization, we introduce multiple style transfer modules in the
decoder and INSLs. PhotoNet significantly outperforms existing algorithms in
terms of both efficiency and effectiveness. In the P-step, we adopt a neural
architecture search method to accelerate PhotoNet. We propose an automatic
network pruning framework in the manner of teacher-student learning for
photorealistic stylization. The network architecture named PhotoNAS resulted
from the search achieves significant acceleration over PhotoNet while keeping
the stylization effects almost intact. We conduct extensive experiments on both
image and video transfer. The results show that our method can produce
favorable results while achieving 20-30 times acceleration in comparison with
the existing state-of-the-art approaches. It is worth noting that the proposed
algorithm accomplishes better performance without any pre- or post-processing.
