Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition
  Decomposing a scene into its shape, reflectance and illumination is a
fundamental problem in computer vision and graphics. Neural approaches such as
NeRF have achieved remarkable success in view synthesis, but do not explicitly
perform decomposition and instead operate exclusively on radiance (the product
of reflectance and illumination). Extensions to NeRF, such as NeRD, can perform
decomposition but struggle to accurately recover detailed illumination, thereby
significantly limiting realism. We propose a novel reflectance decomposition
network that can estimate shape, BRDF, and per-image illumination given a set
of object images captured under varying illumination. Our key technique is a
novel illumination integration network called Neural-PIL that replaces a costly
illumination integral operation in the rendering with a simple network query.
In addition, we also learn deep low-dimensional priors on BRDF and illumination
representations using novel smooth manifold auto-encoders. Our decompositions
can result in considerably better BRDF and light estimates enabling more
accurate novel view-synthesis and relighting compared to prior art. Project
page: https://markboss.me/publication/2021-neural-pil/
