Unsupervised Learning of Depth and Depth-of-Field Effect from Natural
  Images with Aperture Rendering Generative Adversarial Networks
  Understanding the 3D world from 2D projected natural images is a fundamental
challenge in computer vision and graphics. Recently, an unsupervised learning
approach has garnered considerable attention owing to its advantages in data
collection. However, to mitigate training limitations, typical methods need to
impose assumptions for viewpoint distribution (e.g., a dataset containing
various viewpoint images) or object shape (e.g., symmetric objects). These
assumptions often restrict applications; for instance, the application to
non-rigid objects or images captured from similar viewpoints (e.g., flower or
bird images) remains a challenge. To complement these approaches, we propose
aperture rendering generative adversarial networks (AR-GANs), which equip
aperture rendering on top of GANs, and adopt focus cues to learn the depth and
depth-of-field (DoF) effect of unlabeled natural images. To address the
ambiguities triggered by unsupervised setting (i.e., ambiguities between smooth
texture and out-of-focus blurs, and between foreground and background blurs),
we develop DoF mixture learning, which enables the generator to learn real
image distribution while generating diverse DoF images. In addition, we devise
a center focus prior to guiding the learning direction. In the experiments, we
demonstrate the effectiveness of AR-GANs in various datasets, such as flower,
bird, and face images, demonstrate their portability by incorporating them into
other 3D representation learning GANs, and validate their applicability in
shallow DoF rendering.
