Pix2NeRF: Unsupervised Conditional $\pi$-GAN for Single Image to Neural
  Radiance Fields Translation
  We propose a pipeline to generate Neural Radiance Fields~(NeRF) of an object
or a scene of a specific class, conditioned on a single input image. This is a
challenging task, as training NeRF requires multiple views of the same scene,
coupled with corresponding poses, which are hard to obtain. Our method is based
on $\pi$-GAN, a generative model for unconditional 3D-aware image synthesis,
which maps random latent codes to radiance fields of a class of objects. We
jointly optimize (1) the $\pi$-GAN objective to utilize its high-fidelity
3D-aware generation and (2) a carefully designed reconstruction objective. The
latter includes an encoder coupled with $\pi$-GAN generator to form an
auto-encoder. Unlike previous few-shot NeRF approaches, our pipeline is
unsupervised, capable of being trained with independent images without 3D,
multi-view, or pose supervision. Applications of our pipeline include 3d avatar
generation, object-centric novel view synthesis with a single input image, and
3d-aware super-resolution, to name a few.
