Towards Modality Transferable Visual Information Representation with
  Optimal Model Compression
  Compactly representing the visual signals is of fundamental importance in
various image/video-centered applications. Although numerous approaches were
developed for improving the image and video coding performance by removing the
redundancies within visual signals, much less work has been dedicated to the
transformation of the visual signals to another well-established modality for
better representation capability. In this paper, we propose a new scheme for
visual signal representation that leverages the philosophy of transferable
modality. In particular, the deep learning model, which characterizes and
absorbs the statistics of the input scene with online training, could be
efficiently represented in the sense of rate-utility optimization to serve as
the enhancement layer in the bitstream. As such, the overall performance can be
further guaranteed by optimizing the new modality incorporated. The proposed
framework is implemented on the state-of-the-art video coding standard (i.e.,
versatile video coding), and significantly better representation capability has
been observed based on extensive evaluations.
