Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff
  Lossy compression algorithms are typically designed and analyzed through the
lens of Shannon's rate-distortion theory, where the goal is to achieve the
lowest possible distortion (e.g., low MSE or high SSIM) at any given bit rate.
However, in recent years, it has become increasingly accepted that "low
distortion" is not a synonym for "high perceptual quality", and in fact
optimization of one often comes at the expense of the other. In light of this
understanding, it is natural to seek for a generalization of rate-distortion
theory which takes perceptual quality into account. In this paper, we adopt the
mathematical definition of perceptual quality recently proposed by Blau &
Michaeli (2018), and use it to study the three-way tradeoff between rate,
distortion, and perception. We show that restricting the perceptual quality to
be high, generally leads to an elevation of the rate-distortion curve, thus
necessitating a sacrifice in either rate or distortion. We prove several
fundamental properties of this triple-tradeoff, calculate it in closed form for
a Bernoulli source, and illustrate it visually on a toy MNIST example.
