Unsupervised Enhancement of Real-World Depth Images Using Tri-Cycle GAN
  Low quality depth poses a considerable challenge to computer vision
algorithms. In this work we aim to enhance highly degraded, real-world depth
images acquired by a low-cost sensor, for which an analytical noise model is
unavailable. In the absence of clean ground-truth, we approach the task as an
unsupervised domain-translation between the low-quality sensor domain and a
high-quality sensor domain, represented using two unpaired training sets. We
employ the highly-successful Cycle-GAN to this task, but find it to perform
poorly in this case. Identifying the sources of the failure, we introduce
several modifications to the framework, including a larger generator
architecture, depth-specific losses that take into account missing pixels, and
a novel Tri-Cycle loss which promotes information-preservation while addressing
the asymmetry between the domains. We show that the resulting framework
dramatically improves over the original Cycle-GAN both visually and
quantitatively, extending its applicability to more challenging and asymmetric
translation tasks.
